无法腌制<function stringtongrams =“” at =“” 0x104144f28 =“”>：它与__main __。stringToNgrams不同

Question

我正在尝试使用多处理在对象列表上运行功能。但是，此特定功能不不属于类或模块-它直接在多处理代码上方定义。

这是我的代码：

def stringToNgrams(string,n):
    ngrams = set()
    sentenceArray = string.split(" ")
    for i in range(0,len(sentenceArray) - (n - 1)):
        ngramWords = sentenceArray[i:i + n]
        print(ngramWords)
        ngram = " ".join(ngramWords)
        print(ngram)
        ngrams.add(ngram)

from multiprocessing import Pool
ngramPool = Pool(3)
import functools
stringToNgrams = functools.partial(stringToNgrams,n=2) # So we can load n into stringToNgrams.
results = ngramPool.map(stringToNgrams,strings)

这是我的错误消息：

  File "/Users/montana/Documents/Capstone project/BBEditRunTemp-n_gram.py", line 76, in <module>
    results = ngramPool.map(stringToNgrams,strings)
  File "/Users/montana/miniconda3/lib/python3.6/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Users/montana/miniconda3/lib/python3.6/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/Users/montana/miniconda3/lib/python3.6/multiprocessing/pool.py", line 385, in _handle_tasks
    put(task)
  File "/Users/montana/miniconda3/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/Users/montana/miniconda3/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function stringToNgrams at 0x104144f28>: it's not the same object as __main__.stringToNgrams

我似乎不知道该怎么做-如果使用 main .stringToNgrams，Python会抱怨未定义名称“ main ”。请帮助我，谢谢！

Answer 1

似乎这是一个与functools.partial构造函数有关的问题。来自the Python bug tracker：

  

libric functools.partial   构造函数很难正确地腌制部分实例。

因此，答案是要做两件事之一：

1）使n为可选参数

2）创建另一个将2传递给原始函数的函数。