Question

好的，所以我个人觉得这很奇怪，但我希望对此有个解决办法：我目前正在python中进行多处理实验，而我的基本设置是我有多个线程正在同时做事（当前只是对数字进行平方运算），然后将其结果输出到Queue中，以便该Queue的末尾有一个Process处理信息（将这些项目放入列表中），如下所示：

from multiprocessing import Process, Manager, Pool as ThreadPool


class SessionsPar:
    def __init__(self, read_queue, write_items):
        self.task_done = "DONE"
        self.queue = read_queue
        self.dfs = write_items

    def write_queue(self, msg):
        self.queue.put(msg)

    def read_queue(self):
        while True:
            msg = self.queue.get()
            if isinstance(msg, str) and msg == self.task_done:
                break
            self.dfs.append(msg)

    def gen_stuff(self, num):
        self.write_queue(num * num)

    def run(self):
        reader = Process(target=self.read_queue)
        reader.start()
        pool = ThreadPool(4)
        pool.map(self.gen_stuff, range(550))
        pool.close()
        pool.join()
        self.write_queue(self.task_done)


manager = Manager()
queue = manager.Queue()
items = manager.list()
sess = SessionsPar(read_queue=queue, write_items=items)
sess.run()
print("sess.dfs: {}".format(sess.dfs))

我只是创建一个数字列表，将它们平方，然后将它们放入队列中，然后让进程从那里读取内容，顺序无关紧要。现在来了怪异的部分。当我重复运行代码时，会发生以下情况：

代码运行得很好，并在末尾给出了平方数的列表

与1相同，但具有以下Stacktrace和Errormessage：

Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "<path>/time/parallel_queues.py", line 15, in read_queue
    msg = self.queue.get()
  File "<string>", line 2, in get
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
    conn.send((self._id, methodname, args, kwds))
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header + buf)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

与1相同，但具有以下 Stacktrace和Errormessage：

Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "<path>/time/parallel_queues.py", line 18, in read_queue
    self.dfs.append(msg)
  File "<string>", line 2, in append
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod
    kind, result = conn.recv()
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

只需运行几次而无需进行任何更改，就会得到上面的结果。搜索并没有多大帮助，因为对于BrokenPipes，我发现所有内容都与I / O有关，而对于第二条消息，这发生在网络环境中，而这根本不是我要做的。

编辑：如果有帮助：我正在Linux机器上运行Python 3.6.6，[GCC 8.1.1]。

对等点的Python多处理队列中断管道/连接重置

0 个答案: