Python多处理过程:使用Manager Queue时识别事件的延迟

时间:2017-09-05 09:01:46

标签: python events multiprocessing

我有以下代码:

from multiprocessing import Process, Manager, Event

manager = Manager()
shared_Queue = manager.Queue(10)
ev = Event()

def do_this(shared_queue, ev):
    while not ev.is_set():
        if not shared_Queue.__getattribute__('empty')():
           item = shared_queue.get()
           print item
    print 'released!'

subprocs = []
for i in xrange(10):
    subproc = Process(target=do_this, args=(shared_Queue, ev, ))
    subprocs.append(subproc)
    subproc.start()

现在,如果我运行它,我会问这些进程是否还活着:

for subproc in subprocs: print subproc.is_alive()

当然,我得到了所有的真理。

经过几次这样做:*如果我不做这些就没有错误!

shared_Queue.put(3)
shared_Queue.put(5)

现在我想使用以下命令设置事件以杀死所有这些:

ev.set()

但后来没有看到'发布!' 10次​​,我得到不同数量的这些打印件,大约2到5秒后,我得到了一堆错误:

released!
released!
released!
released!
released!
released!
released!
Process Process-10:
Traceback (most recent call last):
  File 
 "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "<input>", line 10, in do_this
  File "<string>", line 2, in get
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/managers.py", line 759, in _callmethod
    kind, result = conn.recv()
EOFError
Process Process-5:
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "<input>", line 10, in do_this
  File "<string>", line 2, in get
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/managers.py", line 759, in _callmethod
    kind, result = conn.recv()
EOFError
Process Process-7:
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "<input>", line 10, in do_this
  File "<string>", line 2, in get
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/managers.py", line 759, in _callmethod
    kind, result = conn.recv()
EOFError

为什么某些进程无法识别事件集并在以后显示为错误?是否有更好的方式来表明他们死亡?

1 个答案:

答案 0 :(得分:0)

感谢评论stovfl,你是对的,ev.set()不会杀死任何我不小心使用这个词的东西。

至于我遇到的问题,我了解到多处理队列是进程和线程安全的,这意味着,如果队列已经满了,我的进程将在写入队列之前暂停。

如果我尝试设置事件,而某些进程仍在等待向完整队列写入内容,则他们将无法识别事件集。

关键是要清空所有队列,让进程写入它,然后到达可以检查事件的第一行!