如何通过Python访问Hive?

https://cwiki.apache.org/confluence/display/Hive/HiveClient#HiveClient-Python似乎已经过时了。

当我将它添加到/ etc / profile中:

export PYTHONPATH=$PYTHONPATH:/usr/lib/hive/lib/py

然后我可以做链接中列出的进口,除了hive import ThriftHive,其实是需要的:

from hive_service import ThriftHive

接下来,例子中的端口是10000,当我尝试导致程序挂起。 Hive Thrift端口的默认端口为9083,停止挂起。

所以我设置如下:

from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
try:
    transport = TSocket.TSocket('<node-with-metastore>', 9083)
    transport = TTransport.TBufferedTransport(transport)
    protocol = TBinaryProtocol.TBinaryProtocol(transport)
    client = ThriftHive.Client(protocol)
    transport.open()
    client.execute("CREATE TABLE test(c1 int)")

    transport.close()
except Thrift.TException, tx:
    print '%s' % (tx.message)

我收到以下错误:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/hive/lib/py/hive_service/ThriftHive.py", line 68, in execute
self.recv_execute()
File "/usr/lib/hive/lib/py/hive_service/ThriftHive.py", line 84, in recv_execute
raise x
thrift.Thrift.TApplicationException: Invalid method name: 'execute'

但是检查ThriftHive.py文件会显示在Client类中执行的方法。

我如何使用Python访问Hive?

我断言你正在使用HiveServer2,这是使代码不起作用的原因。

您可以使用pyhs2正确访问您的Hive和示例代码:

import pyhs2

with pyhs2.connect(host='localhost',
               port=10000,
               authMechanism="PLAIN",
               user='root',
               password='test',
               database='default') as conn:
    with conn.cursor() as cur:
        #Show databases
        print cur.getDatabases()

        #Execute query
        cur.execute("select * from table")

        #Return column info from query
        print cur.getSchema()

        #Fetch table results
        for i in cur.fetch():
            print i

在安装pyhs2之前,可以先安装python-devel.x86_64 cyrus-sasl-devel.x86_64。

希望这可以帮助你。

参考:https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-PythonClientDriver

http://stackoverflow.com/questions/21370431/how-to-access-hive-via-python

本站文章除注明转载外,均为本站原创或编译
转载请明显位置注明出处:如何通过Python访问Hive?