Connecting to Kerberized hadoop cluster using python module impyla

问题

I am using impyla module to connect to kerberized hadoop cluster. I want to access
hiveserver2/hive but I was getting the below error:

test_conn.py

from impala.dbapi import connect
import os
connection_string = 'hdp296m1.XXX.XXX.com'
conn = connect(host=connection_string, port=21050,auth_mechanism="GSSAPI",kerberos_service_name='testuser@Myrealm.COM',password='testuser')
cursor = conn.cursor()
cursor.execute('select count(*) form t_all_types_simple_t')
print cursor.description
results = cursor.fetchall()

Stacktrace:

[vagrant@localhost vagrant]$ python test_conn.py
Traceback (most recent call last):
  File "test_conn.py", line 4, in <module>
    conn = connect(host=connection_string, port=21050, auth_mechanism="GSSAPI",kerberos_service_name='testuser@Myrealm.COM',password='testuser')
  File "/usr/lib/python2.7/site-packages/impala/dbapi.py", line 147, in connect
    auth_mechanism=auth_mechanism)
  File "/usr/lib/python2.7/site-packages/impala/hiveserver2.py", line 758, in connect
    transport.open()
  File "/usr/lib/python2.7/site-packages/thrift_sasl/__init__.py", line 61, in open
    self._trans.open()
  File "/usr/lib64/python2.7/site-packages/thrift/transport/TSocket.py", line 101, in open
    message=message)
thrift.transport.TTransport.TTransportException: Could not connect to hdp296m1.XXX.XXX.com:21050

testuser is my kerberos principal which I will be using to do kinit.

回答1:

Your connection appears to be incorrect.. Try,

from impala.dbapi import *
import sys, os
# set your parms
host=os.environ.get("CDH_HIVE",'x.x.x.x')
port=os.environ.get("CDH_HIVE_port",'10000')
auth_mechanism=os.environ.get("CDH_auth",'GSSAPI')
user='hive' 
db='mydb' 
# No password use kinit 
password=''
# hive is principal with krb
kbservice='hive'  

class Hive:

    def __init__(self,db):
        self.database=db
        self.__conn = connect(host=host,
                            port=port,
                            auth_mechanism=auth_mechanism,
                            user=user,
                            password=password,
                            database=db,
                            kerberos_service_name=kbservice
                            )


        self.__cursor = self.__conn.cursor()


h = Hive(db)

来源：https://stackoverflow.com/questions/41738526/connecting-to-kerberized-hadoop-cluster-using-python-module-impyla

标签

python

python-2.7

Hadoop

kerberos

impyla