Question

我有一个队列，其中有100个数字，从1到100。首先，我有一个进程来填充打印Queue filled的队列。接下来，我有两个函数可以打印队列的当前值。我正在尝试在进程之间打印队列的值之间进行权衡。这是我的代码：

import multiprocessing as mp

def fillQueue(lookup,q):
    list(map(q.put,lookup))
    print('Queue filled')

def printQueue1(q):
    while not q.empty():
        print('Process 1:', (q.get()))
    print('Process 1: Queue is empty!')

def printQueue2(q):
    while not q.empty():
        print('Process 2:', (q.get()))
    print('Process 2: Queue is empty!')

if __name__ == "__main__":
    pool = mp.Pool(processes=3)
    manager = mp.Manager()
    q = manager.Queue()

    lookup = []
    count = 1
    while count < 101:
        lookup.append(count)
        count = count + 1

    p2 = pool.apply_async(printQueue1,(q,))
    p3 = pool.apply_async(printQueue2,(q,))
    p1 = pool.apply_async(fillQueue,(lookup,q))

    pool.close()
    pool.join()

这给了我

Process 1: 1
Process 1: 2
Process 1: 3
Process 1: 4
Process 1: 5
Process 2: 6
Process 1: 7
Process 2: 8
Process 1: 9
Process 2: 10

我想要得到的是：

Queue filled
Process 1: 1
Process 2: 2
Process 1: 3
Process 2: 4
Process 1: 5

任何想法如何实现这一目标？每次运行程序时，都会得到不同的结果，所以发生了一些奇怪的事情。谢谢！

Answer 1

因此，apply_async 异步应用了进程-这意味着您触发运行的3个进程都同时运行，并且相互竞争。

>

由于您不是确定性地触发这些过程，因此，每次触发该过程时，它们的运行顺序可能会发生变化。

我假设你想要

在进程尝试访问之前要填充的队列
“工作”在流程之间平均分配

即使如此，除非您以某种方式限制功能，否则它们将get()个项目的顺序仍然是相当随机的。如果您确实需要function1仅获得赔率，而function2仅获得偶数，并且由于它们处于严格顺序，则您可能不希望进行多重处理...

import multiprocessing as mp


def fillQueue(lookup, q):
    list(map(q.put, lookup))
    print('Queue filled')


def printQueue(q, id):
    while not q.empty():
        print('Process {}: {}'.format(id, q.get()))
    print('Process {}: Queue is empty!'.format(id))


if __name__ == "__main__":
    pool = mp.Pool(processes=3)
    manager = mp.Manager()
    q = manager.Queue()

    # no need to construct a list with a counter, we can just use the generator
    lookup = range(101)

    # do not fill the queue while processes are running, do it beforehand!
    fillQueue(lookup, q)

    # don't need different functions, since they are doing the same work
    # just fire off multiple copies of the same function
    p1 = pool.apply_async(printQueue, (q, 1,))
    p2 = pool.apply_async(printQueue, (q, 2,))

    pool.close()
    pool.join()

示例输出：

Queue filled
Process 2: 0
Process 2: 1
Process 2: 2
Process 2: 3
Process 2: 4
Process 2: 5
Process 1: 6
Process 2: 7
Process 1: 8
Process 2: 9
Process 2: 10
Process 1: 11

Answer 2

您可以为每个进程创建一个Queue对象，以充当“警棍”来指示哪个进程从主队列中取出下一个项目，然后在每个辅助函数的主循环中，应该首先尝试从其自己的“ baton”队列中出队，然后再尝试从主队列中出队，此后，它应通过将一个项目排队到下一个进程的“ baton”队列中来将“警棍”传递到下一个进程。排队过程应通过将一个项目放入应首先运行的过程的“警棍”队列中来启动出列过程。之所以有效，是因为Queue.get一直阻塞到队列中没有项目为止。

import multiprocessing as mp
import time

def fillQueue(lookup, q, baton_first):
    list(map(q.put,lookup))
    print('Queue filled')
    baton_first.put(None)

def printQueue(id, q, baton_self, baton_other):
    while True:
        baton_self.get()
        try:
            if q.empty():
                break
            print('Process %s:' % id, (q.get()))
        # use finally to always pass on the baton whether the loop breaks or not
        finally:
            baton_other.put(None)
        time.sleep(1) # the actual work should be performed here
    print('Process %s: Queue is empty!' % id)

if __name__ == "__main__":
    pool = mp.Pool(processes=3)
    manager = mp.Manager()
    q = manager.Queue()
    baton1 = manager.Queue()
    baton2 = manager.Queue()

    p2 = pool.apply_async(printQueue,(1, q, baton1, baton2))
    p3 = pool.apply_async(printQueue,(2, q, baton2, baton1))
    p1 = pool.apply_async(fillQueue, (list(range(1, 11)), q, baton1))

    pool.close()
    pool.join()

这将输出：

Queue filled
Process 1: 1
Process 2: 2
Process 1: 3
Process 2: 4
Process 1: 5
Process 2: 6
Process 1: 7
Process 2: 8
Process 1: 9
Process 2: 10
Process 1: Queue is empty!
Process 2: Queue is empty!

Python-队列权衡处理

2 个答案: