我正在尝试使用python与hbase throght knox进行交互
管理员为hive,hbase和spark提供了一个knox API端点列表,例如:
https://knox-devl.www.mysite.com:9042/gateway/MYSITEHDO/hbaseversion/cluster
现在,由于我正在使用Python的happybase库,因此我的连接代码为
import happybase
connection=happybase.Connection('https://knox-devl.www.mysite.com/gateway/MYSITEHDO/hbaseversion/cluster',port=9042)
connection.open()
print(connection.tables())
它显示的错误是:
thriftpy.transport.TTransportException: TTransportException(message="Could not connect to ('https://knox-devl.www.mysite.com/gateway/MYSITEHDO/hbaseversion/cluster', 9042)", type=1)
我也尝试了Phoenixdb lib
import phoenixdb
database_url = 'https://knox-devl.www.mysite.com:9042/gateway/MYSITEHDO/hbaseversion/cluster'
conn = phoenixdb.connect(database_url, autocommit=True)
cursor = conn.cursor()
cursor.execute("SHOW tables")
但是我遇到另一个错误:
phoenixdb.errors.InterfaceError: ('RPC request failed', None, None, BadStatusLine("''",))
Exception phoenixdb.errors.InterfaceError: InterfaceError('RPC request failed', None, None, BadStatusLine("''",)) in <bound method Connection.__del__ of <phoenixdb.connection.Connection object at 0x10bc97d90>> ignored
我可以通过curl获取一些数据的唯一方法:
curl -i -k -u guest:guest-password 'https://knox-devl.www.mysite.com:9042/gateway/MYSITEHDO/hbaseversion/cluster'
但是那里没有SQL命令。
有人知道如何做到这一点,或者我在这里缺少什么,例如询问其他URL或在集群上启用某些东西?
答案 0 :(得分:1)
您已经确定,通过Knox与HBase进行通讯的唯一方法是通过HBase的REST API。 Happybase试图通过RPC直接连接到HBase,Knox会阻止它。
您不能在启用了Knox的群集外部使用Happybase。
here可以找到将HBase REST API与Python结合使用的很好的教程。万一链接消失了,本文中一些最有用的命令是:
查看表的模式:
request = requests.get(baseurl + "/" + tablename + "/schema")
插入一行:
cellset = Element('CellSet')
linenumber = 0;
for line in shakespeare:
rowKey = username + "-" + filename + "-" + str(linenumber).zfill(6)
rowKeyEncoded = base64.b64encode(rowKey)
row = SubElement(cellset, 'Row', key=rowKeyEncoded)
messageencoded = base64.b64encode(line.strip())
linenumberencoded = encode(linenumber)
usernameencoded = base64.b64encode(username)
# Add bleet cell
cell = SubElement(row, 'Cell', column=messagecolumnencoded)
cell.text = messageencoded
# Add username cell
cell = SubElement(row, 'Cell', column=usernamecolumnencoded)
cell.text = usernameencoded
# Add Line Number cell
cell = SubElement(row, 'Cell', column=linenumbercolumnencoded)
cell.text = linenumberencoded
linenumber = linenumber + 1
# Submit XML to REST server
request = requests.post(baseurl + "/" + tablename + "/fakerow", data=tostring(cellset), headers={"Content-Type" : "text/xml", "Accept" : "text/xml"})
删除表格:
request = requests.delete(baseurl + "/" + tablename + "/schema")