Question

我在Python 3.4.3中使用多处理来加速我的代码。我收回结果时遇到问题。我尝试了以下简单的代码，工作得很好。

    import numpy
    from multiprocessing import Pool
    from functools import partial
    from OpenDutchWordnet import Wn_grid_parser, le, les, synset, relation

    def funct2(arg1, value):
        return arg1.get_relations(value)

    if __name__ == '__main__':
        myparser= Wn_grid_parser(Wn_grid_parser.odwn)
        l_sensesofwoord = myparser.lemma_get_generator("man")
        sense=l_sensesofwoord[0]
        synsetid_sense=sense.get_synset_id()
        t=["has_hyperonym", "has_holonym"]
        arg1=myparser.synsets_find_synset(synsetid_sense)

        f=partial(funct2, arg1)
        print("f is: ", f)
        m1=pool.map(f,t)

当然，我想运行多个进程。我想要运行的代码如下，

f is:  functools.partial(<function funct2 at 0x00000000046D5378>, <synset.Synset object at 0x000000005011DDA0>) 

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
File "C:\Python34\lib\multiprocessing\pool.py", line 119, in worker
  result = (True, func(*args, **kwds))
File "C:\Python34\lib\multiprocessing\pool.py", line 44, in mapstar
  return list(map(*args))
File "C:\Users\UTRSB\AppData\Local\Continuum\Anaconda3\mycode\multi.py", line 14, in funct2
  return numpy.asarray(arg1.get_relations(value))
File "C:\Python34\lib\site-packages\OpenDutchWordnet\synset.py", line 98, in get_relations
  for relation_el in self.synset_el.iterfind(xml_query)]
File "C:\Python34\lib\site-packages\OpenDutchWordnet\synset.py", line 97, in <listcomp>
  return [Relation(relation_el)
File "C:\Python34\lib\site-packages\lxml\_elementpath.py", line 156, in select
  for elem in result:
File "C:\Python34\lib\site-packages\lxml\_elementpath.py", line 88, in select
  for elem in result:
File "C:\Python34\lib\site-packages\lxml\_elementpath.py", line 89, in select
  for e in elem.iterchildren(tag):
File "lxml.etree.pyx", line 1363, in lxml.etree._Element.iterchildren (src\lxml\lxml.etree.c:50501)
File "lxml.etree.pyx", line 2730, in lxml.etree.ElementChildIterator.__cinit__ (src\lxml\lxml.etree.c:66739)
File "apihelpers.pxi", line 24, in lxml.etree._assertValidNode (src\lxml\lxml.etree.c:14133)
AssertionError: invalid Element proxy at 53353160
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:/Users/UTRSB/AppData/Local/Continuum/Anaconda3/mycode/multi.py", line 52, in <module>
  m1=pool.map(f,t)
File "C:\Python34\lib\multiprocessing\pool.py", line 260, in map
  return self._map_async(func, iterable, mapstar, chunksize).get()
File "C:\Python34\lib\multiprocessing\pool.py", line 599, in get
  raise self._value
AssertionError: invalid Element proxy at 53353160

运行此代码时，我收到以下错误消息。

result=pool.apply_async(geefAlleGloss,[p])

我也尝试过另一种方式：answer=result.get() 这工作得很好，但是当我想使用get（）来获得结果时。我最终得到了同样的错误。 obj

我认为错误是map函数中的某个地方。起初，我认为它与我使用的OpenDutchWordnet导入的模块有关。但是当部分函数工作时，错误应该由get（）和/或map（）函数引起。

我将不胜感激任何帮助。

Python 3.4中的多处理不适用于导入的模块

0 个答案: