Question

我尝试在项目中使用pathos.multiprocessing.Pool。但是，当我终止池时，它将遇到以下问题。我使用的是CentOS 6.5，我不确定它是否是由pathos.multiprocessing.Pool或其他东西引起的，有人可以帮我吗？

  Traceback (most recent call last):
File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  self.run()
File "/usr/local/lib/python2.7/threading.py", line 1073, in run
  self.function(*self.args, **self.kwargs)
File "receiver.py", line 132, in kill_clients
  pool.terminate()
File "/usr/local/lib/python2.7/site-packages/multiprocess/pool.py", line 465, in terminate
  self._terminate()
File "/usr/local/lib/python2.7/site-packages/multiprocess/util.py", line 207, in __call__
  res = self._callback(*self._args, **self._kwargs)
File "/usr/local/lib/python2.7/site-packages/multiprocess/pool.py", line 513, in _terminate_pool
  p.terminate()
File "/usr/local/lib/python2.7/site-packages/multiprocess/process.py", line 137, in terminate
  self._popen.terminate()
File "/usr/local/lib/python2.7/site-packages/multiprocess/forking.py", line 174, in terminate
  os.kill(self.pid, signal.SIGTERM)

OSError：[Errno 3]没有这样的过程

有线的事情是，一开始，它运作良好。但是当收到第4份工作时，就会出现这样的问题。

    class Receiver:
    def __init__(self):
        ....
        self.results={}
    def kill_clients(self, client_list, pool):
        for client in client_list:
            client.kill()
        pool.terminate()
    def process_result(self, result):
        if result is None:
            self.results = {}
            return
        res = result.split(':')
        if len(res) != 4:
            raise Exception("result with wrong format: %s" % result)
        self.results['%s_%s' % (res[0], res[1])] = {"code": res[3], "msg": res[4]}
    ...

     def handler(self, job):
        self.lg.debug("Receive job in rtmp_start_handler.")
        self.lg.debug("<%s>" % str(job))
        # each client corresponding one process
        cli_counts = job['count']
        pool = Pool(processes=cli_counts)
        clients = []
        try:
            for i in xrange(cli_counts):
                rtmp_cli = RtmpClient(job['case'], i)
                clients.append(rtmp_cli)
            [pool.apply_async(client.run, callback=self.process_result)
             for client in clients]
            pool.close()
            sleep(1)
            self.lg.debug("All clients are started.")
            t = Timer(
                job['timeout'],
                self.kill_clients,
                args=(clients, pool)
            )
            t.start()
            self.lg.debug("Timer is started. timeout %s s" % job['timeout'])
            pool.join()
        except Exception, e:
            self.lg.warning("Exception occurred: %s" % e)
            self.lg.warning(format_exc())
            return "0"

        # here the self.results shall be ready
        return self.parse_results()

Answer 1

OSError不是由Pool引起的，而是由我的程序问题引起的。当我使用Popen创建子进程和exec ffmpeg时，它会立即退出（由于其他问题），所以当我尝试杀死子进程时，它就不存在了。这就是为什么会引发OSError的原因。

有时pathos.multiprocessing.Pool无法正确终止

1 个答案: