Question

我正在使用Python 3.7.1将minhash制成字符串列表。代码如下。

import mmh3
import random
import string
import itertools
from datasketch import MinHash

def grouper(iterable,n=2):
    return ["".join(x) for x in list(itertools.permutations(iterable, n))]

def _hash_func(d):
    return mmh3.hash(d)

def _run_minhash(data, seed):

    m = MinHash(num_perm=128 ,hashfunc=_hash_func)
    for d in data:
        m.update(d.encode('utf8'))
    return m.count()

if __name__ == '__main__':
    st = string.ascii_uppercase
    ngrams  = grouper(st,n=4)
    print(_run_minhash(ngrams,seed=12))

要获得加速，我正在使用datasketch.Minhash documentation中提到的mmh3.hash（mmh3.hash32不再可用），它会产生以下错误

TypeError：输入类型不支持ufunc'bitwise_and'，并且根据转换规则“安全”，不能将输入安全地强制转换为任何受支持的类型

Traceback (most recent call last):
  File "hashes.py", line 41, in 
    print(_run_minhash(ngrams,seed=12))
  File "hashes.py", line 35, in _run_minhash
    m.update(d.encode('utf8'))
  File "/home/nithin/miniconda3/lib/python3.7/site-packages/datasketch/minhash.py", line 134, in update
    phv = np.bitwise_and((a * hv + b) % _mersenne_prime, np.uint64(_max_hash))
TypeError: ufunc 'bitwise_and' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

我没有找到具体的原因以及解决方法。我该如何解决？还是有其他方法加快minhash计算。在此先感谢您的宝贵时间。如果我错过了任何事情，请在评论中让我知道。

输入类型Minhash不支持ufunc'bitwise_and'

0 个答案: