Question

我试图写一个test to compare methods for inter process communication in python，OSX和windows给我的第一个样本问题。它几乎是使用队列的手动示例。

我使用python 2.7.9和3.4.1在linux上编写了所有代码。它工作正常。

#!/usr/bin/env python
import multiprocessing as MP
...
self.mpq_main = MP.Queue()
self.mpq_net = MP.Queue()
self.mpt_net = MP.Process( target=self.start_t_net, args=(self.mpq_main, self.mpq_net) )
...
def start_t_net(self, qin, qout):
    self.NET = gcbtestnet.GCBTestNET(qin, qout);

然后在osx和windows上测试我得到错误。这个问题适用于windows one：

C:\Documents and Settings\user1\Desktop>C:\Python27\python.exe gcbtest.py
Traceback (most recent call last):
  File "gcbtest.py", line 82, in <module>
    theclass = GCBTest()
  File "gcbtest.py", line 31, in __init__
    self.mpt_fs.start();
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\multiprocessing\forking.py", line 67, in dispatcher
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 401, in save_reduce
    save(args)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 548, in save_tuple
    save(element)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 725, in save_inst
    save(stuff)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 313, in save
    (t.__name__, obj))
pickle.PicklingError: Can't pickle '_subprocess_handle' object: <_subprocess_han
dle object at 0x00ABD190>
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1378, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 880, in load_eof
    raise EOFError
EOFError
UI check
UI check
UI check
...

如果您尝试将对象引用作为进程参数传递，则通常会显示该错误，但我只传递multiprocess.Queues个实例。

Answer 1

我发现了我的（粗暴）错误。当我将进程从简单方法移动到对象时，我忘记将所述对象的初始化移动到.Process()调用并将其连接到类方法中。出于某种原因，它适用于Linux，但不适用于Windows。

这是带有修复https://github.com/gcb/python_multiprocess_test/commit/8565384e5cc6cadf959c335b5db0693e646f6777

的差异

-self.mpt_net = MP.Process( target=self.start_t_net, args=(self.mpq_main, self.mpq_net) )
+self.mpt_net = MP.Process( target=gcbtestnet.GCBTestNET, args=(self.mpq_main, self.mpq_net) )
...
-def start_t_net(self, qin, qout):
-    self.NET = gcbtestnet.GCBTestNET(qin, qout);

pickle.PicklingError：不能腌制＆＃39; _subprocess_handle＆＃39; object：＆lt; _subprocess_han dle object at 0x00AAAAAAA＆gt;

1 个答案: