python中的多进程:跨进程来处理watch-kill-restart

时间:2013-06-17 09:53:08

标签: python multiprocessing restart watch

想要对同一父母开始的两个进程进行“交叉生活”控制。

如果进程A花费太长时间来更改进程B的读取列表中的值,则进程B应该终止并重新启动A,反之亦然。

这是从父级启动进程的方式:

def check():
    ....# ok check the value in the R list
    #kill and restart

def A:
    check()#kill and restart B if something goes wrong
    #... do his homework


def B:
    check()#kill and restart A if something goes wrong
    #... do his homework

job1 = multiprocessing.Process(target=A, args=(R,W))
job2 = multiprocessing.Process(target=B, args=(W,R))


procs = []
procs.append(job1)
procs.append(job2)



if __name__ == '__main__':

    _start = time.time()
    map(lambda x: x.start(), procs)                                
    #job1.terminate() it kill the process but can't call it from inside A or B
    map(lambda x: x.join(), procs)

A和B都调用(并且这项工作)到父项中定义的函数并查找列表中的更改但是如何从运行的该函数调用terminate()和run()在进程内部因此无法知道其他进程的名称,pid?

2 个答案:

答案 0 :(得分:0)

我启用了Debug函数,并试图从父级中终止并重启子进程。

我在类中转换了A和B并添加了一个关闭失败进程的关闭方法,在此之后,如果我尝试A.start()或A.run(),则会出现此错误“无法启动进程两次”这意味着什么?这个过程已经死了,没有其他人在运行。

如果我调用multiprocessing.active_childre(),则不会重新启动该进程但会得到此输出:

 <Process(SyncManager-2, started)>, <Process(SyncManager-1, started)>, <Process(SyncManager-3, started)>]
[INFO/MainProcess] process shutting down
[DEBUG/MainProcess] running all "atexit" finalizers with priority >= 0
[SUBDEBUG/MainProcess] calling <Finalize object, callback=_decref, args=(Token(typeid='Value', address='/tmp/pymp-2ObnP1/listener-aIRbnG', id='7f28db9b6190'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6718>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98da10>, ProcessLocalSet(['7f28db9b6190']), <function Client at 0x7f28db99b9b0>), exitprority=10>
[SUBDEBUG/MainProcess] finalizer calling <function _decref at 0x7f28db99faa0> with args (Token(typeid='Value', address='/tmp/pymp-2ObnP1/listener-aIRbnG', id='7f28db9b6190'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6718>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98da10>, ProcessLocalSet(['7f28db9b6190']), <function Client at 0x7f28db99b9b0>) and kwargs {}
[DEBUG/MainProcess] DECREF '7f28db9b6190'
[DEBUG/MainProcess] thread 'Dummy-1' has no more proxies so closing conn
[SUBDEBUG/MainProcess] calling <Finalize object, callback=_decref, args=(Token(typeid='Value', address='/tmp/pymp-EfUPHj/listener-hPdyzk', id='7f28db9b30d0'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6638>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98d9a8>, ProcessLocalSet(['7f28db9b30d0']), <function Client at 0x7f28db99b9b0>), exitprority=10>
[SUBDEBUG/MainProcess] finalizer calling <function _decref at 0x7f28db99faa0> with args (Token(typeid='Value', address='/tmp/pymp-EfUPHj/listener-hPdyzk', id='7f28db9b30d0'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6638>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98d9a8>, ProcessLocalSet(['7f28db9b30d0']), <function Client at 0x7f28db99b9b0>) and kwargs {}
[DEBUG/MainProcess] DECREF '7f28db9b30d0'
[DEBUG/MainProcess] thread 'Dummy-1' has no more proxies so closing conn
[SUBDEBUG/MainProcess] calling <Finalize object, callback=_decref, args=(Token(typeid='list', address='/tmp/pymp-W6FZ5G/listener-V4fUuw', id='7f28db9b3908'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28ddc2f7f8>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98d940>, ProcessLocalSet(['7f28db9acf38', '7f28db9b3908']), <function Client at 0x7f28db99b9b0>), exitprority=10>
[SUBDEBUG/MainProcess] finalizer calling <function _decref at 0x7f28db99faa0> with args (Token(typeid='list', address='/tmp/pymp-W6FZ5G/listener-V4fUuw', id='7f28db9b3908'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28ddc2f7f8>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98d940>, ProcessLocalSet(['7f28db9acf38', '7f28db9b3908']), <function Client at 0x7f28db99b9b0>) and kwargs {}
[DEBUG/MainProcess] DECREF '7f28db9b3908'
[SUBDEBUG/MainProcess] calling <Finalize object, callback=_decref, args=(Token(typeid='list', address='/tmp/pymp-W6FZ5G/listener-V4fUuw', id='7f28db9acf38'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28ddc2f7f8>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98d940>, ProcessLocalSet(['7f28db9acf38']), <function Client at 0x7f28db99b9b0>), exitprority=10>
[SUBDEBUG/MainProcess] finalizer calling <function _decref at 0x7f28db99faa0> with args (Token(typeid='list', address='/tmp/pymp-W6FZ5G/listener-V4fUuw', id='7f28db9acf38'), '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28ddc2f7f8>, <multiprocessing.util.ForkAwareLocal object at 0x7f28db98d940>, ProcessLocalSet(['7f28db9acf38']), <function Client at 0x7f28db99b9b0>) and kwargs {}
[DEBUG/MainProcess] DECREF '7f28db9acf38'
[DEBUG/MainProcess] thread 'Dummy-1' has no more proxies so closing conn
[SUBDEBUG/MainProcess] calling <Finalize object, callback=_finalize_manager, args=(<Process(SyncManager-3, started)>, '/tmp/pymp-2ObnP1/listener-aIRbnG', '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6718>, <function Client at 0x7f28db99b9b0>), exitprority=0>
[SUBDEBUG/MainProcess] finalizer calling <function _finalize_manager at 0x7f28db99f5f0> with args (<Process(SyncManager-3, started)>, '/tmp/pymp-2ObnP1/listener-aIRbnG', '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6718>, <function Client at 0x7f28db99b9b0>) and kwargs {}
[INFO/MainProcess] sending shutdown message to manager
[SUBDEBUG/MainProcess] calling <Finalize object, callback=_finalize_manager, args=(<Process(SyncManager-2, started)>, '/tmp/pymp-EfUPHj/listener-hPdyzk', '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6638>, <function Client at 0x7f28db99b9b0>), exitprority=0>
[SUBDEBUG/MainProcess] finalizer calling <function _finalize_manager at 0x7f28db99f5f0> with args (<Process(SyncManager-2, started)>, '/tmp/pymp-EfUPHj/listener-hPdyzk', '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28db9a6638>, <function Client at 0x7f28db99b9b0>) and kwargs {}
[INFO/MainProcess] sending shutdown message to manager
[SUBDEBUG/MainProcess] calling <Finalize object, callback=_finalize_manager, args=(<Process(SyncManager-1, started)>, '/tmp/pymp-W6FZ5G/listener-V4fUuw', '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28ddc2f7f8>, <function Client at 0x7f28db99b9b0>), exitprority=0>
[SUBDEBUG/MainProcess] finalizer calling <function _finalize_manager at 0x7f28db99f5f0> with args (<Process(SyncManager-1, started)>, '/tmp/pymp-W6FZ5G/listener-V4fUuw', '\xc0\xad;K\xce2\xb1\x0f\x17\xe4|f\x14\x1a\x11\xc0\x89 n\x00\x00.\x19B0\xcd%\xac \x83\x8e\x87', <multiprocessing.managers.State object at 0x7f28ddc2f7f8>, <function Client at 0x7f28db99b9b0>) and kwargs {}
[INFO/MainProcess] sending shutdown message to manager
[DEBUG/MainProcess] running the remaining "atexit" finalizers

答案 1 :(得分:0)

如果我使用multiprocessing.Manager杀死一个共享列表的进程,那么有没有办法将所有值设置为“0”并防止其他程序崩溃以寻找相同的共享列表?

如果我终止共享列表然后重新启动它的进程,这是控制台中的输出:

parent process: 19416
process id: 19500
[DEBUG/gigi-5:20] thread 'MainThread' does not own a connection
[DEBUG/gigi-5:20] making connection to manager
[DEBUG/gigi-5:20] failed to connect to address /tmp/pymp-0HGU9Z/listener-HVxKEA
[INFO/gigi-5:20] process shutting down
[DEBUG/gigi-5:20] running all "atexit" finalizers with priority >= 0
[DEBUG/gigi-5:20] running the remaining "atexit" finalizers
Process Pippo-5:20:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/gigi/TESTCOM/gigi.py", line 62, in run
    COM.pid2.value=COM.info('gigi')                                  # assegna a pid1 il PID del processo
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 1040, in set
    return self._callmethod('set', (value,))
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 755, in _callmethod
    self._connect()
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 742, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/usr/lib/python2.7/multiprocessing/connection.py", line 169, in Client
    c = SocketClient(address)
  File "/usr/lib/python2.7/multiprocessing/connection.py", line 297, in SocketClient
    s.connect(address)
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory
[INFO/gigi-5:20] process exiting with exitcode 1