使用happybase扫描具有中文字符的HBase表时,过滤器不起作用

时间:2017-07-18 04:26:37

标签: python hbase thrift happybase

我在HBase中有一个表格,其中的中文字符存储在某个列中,例如< FLT:CREW_DEPT'。现在我需要过滤掉所有的行' FLT:CREW_DEPT'等于某个值。在hbase shell中执行此操作时,它工作正常,如下所示

scan 'PAX_EXP_FACT', {COLUMNS => ['FLT:CREW_DEPT'], FILTER => "SingleColumnValueFilter ('FLT', 'CREW_DEPT', =, 'binary:\xe4\xb8\x80\xe9\x83\xa8', true, true)", LIMIT => 5}
ROW                                                  COLUMN+CELL
 CA101-20160808-PEK-001192753702                     column=FLT:CREW_DEPT, timestamp=1500346136328, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161103-PEK-001181988752                     column=FLT:CREW_DEPT, timestamp=1500346230204, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161105-PEK-000728690130                     column=FLT:CREW_DEPT, timestamp=1500346244963, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161201-PEK-006731936575                     column=FLT:CREW_DEPT, timestamp=1500346233640, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161212-PEK-001512808262                     column=FLT:CREW_DEPT, timestamp=1500346223572, value=\xE4\xB8\x80\xE9\x83\xA8
5 row(s) in 0.0060 seconds

然而,当使用happybase在python中执行类似的操作时,没有返回任何内容:

import happybase
import datetime
import pytz

connection = happybase.Connection('192.168.199.200', port=9090)
table = connection.table('PAX_EXP_FACT')

filter_str = ""
filter_str += "SingleColumnValueFilter('FLT', 'CREW_DEPT', =, 'binary:\xe4\xba\x8c\xe9\x83\xa8')"

results = table.scan(
    filter=filter_str
    #     ,limit=100
)

count = 0
for key, data in results:
    count += 1
    print(data[b'FLT:CREW_DEPT'].decode('utf-8'))

print('No. of flight matches:', count)

connection.close()

返回0行......

有人可以帮忙吗?非常感谢!!!

1 个答案:

答案 0 :(得分:1)

答案结果非常简单......我应该已经

"SingleColumnValueFilter('FLT', 'CREW_DEPT', =, 'binary:中文')"

而不是首先将它转换为utf-8编码的字节串...即使在hbase shell中我也可以做同样的事情(尽管它在问号中显示)

scan 'PAX_EXP_FACT', {COLUMNS => ['FLT:CREW_DEPT'], FILTER => "SingleColumnValueFilter ('FLT', 'CREW_DEPT', =, 'binary:??', true, true)", LIMIT => 5}
ROW                                                  COLUMN+CELL
 CA101-20160808-PEK-001192753702                     column=FLT:CREW_DEPT, timestamp=1500353334419, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161103-PEK-001181988752                     column=FLT:CREW_DEPT, timestamp=1500353426641, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161105-PEK-000728690130                     column=FLT:CREW_DEPT, timestamp=1500353447707, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161201-PEK-006731936575                     column=FLT:CREW_DEPT, timestamp=1500353432222, value=\xE4\xB8\x80\xE9\x83\xA8
 CA101-20161212-PEK-001512808262                     column=FLT:CREW_DEPT, timestamp=1500353417107, value=\xE4\xB8\x80\xE9\x83\xA8
5 row(s) in 0.0100 seconds

使用子字符串不起作用,这是我在提出这个愚蠢的问题之前一直在尝试的......