Question

使用带有进程池的队列（即

）似乎是一种常见的模式。

Pool(2).map(f, xs)

但f的主体可能会附加到要映射的项上，例如

from multiprocessing import Pool

xs = [0]

def f(n):
    global xs
    if n < 10:
        xs.append(n + 1)
    return n

Pool(2).map(f, xs)

期望返回[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

我意识到可以使用mt提供的原语来构建此“ manually”，但是看来这是一个足够普遍的模式，必须有一个通用的解决方案。你知道一个吗？

Answer 1

根据@martineau的建议，您的代码可以更新为：

import multiprocessing as mp


def f(n, xs, xn):
    if n < 10:
        xn.append(n)
        xs.append(n + 1)
        xn.append(n)
        xs.append(n + 2)


if __name__ == '__main__':
    with mp.Manager() as manager:
        xs = manager.list()
        xn = manager.list()
        with mp.Pool(processes=2) as pool:
            pool.starmap(f, [(n, xs, xn) for n in range(20)])
        print(xn)
        print(xs)

此打印

[3, 0, 3, 0, 4, 1, 4, 1, 5, 2, 5, 2, 6, 6, 7, 9, 7, 9, 8, 8]
[4, 1, 5, 2, 5, 2, 6, 3, 6, 3, 7, 4, 7, 8, 8, 10, 9, 11, 9, 10]

您看到的情况并不能保证保留n个值的生成顺序。

编辑：

import multiprocessing as mp


def f(n):
    thresh = 10
    if max(xs) <= thresh and n < thresh:
        xs.append(n + 1)


if __name__ == '__main__':
    with mp.Manager() as manager:
        xs = manager.list([0])
        with mp.Pool(processes=2) as pool:
            pool.map(f, range(20))
        print(sorted(xs))

这幅画

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Answer 2

您可以创建一个类，该类可以通过类似以下的原语来实现：

int[] attrs = new int[]{R.attr.selectableItemBackground};
TypedArray typedArray = this.obtainStyledAttributes(attrs);
int selectableItemBackground = typedArray.getResourceId(0, 0);
typedArray.recycle();

cardView.setForeground(this.getDrawable(selectableItemBackground));
cardView.setClickable(true);

要测试它，请运行-

from multiprocessing import JoinableQueue, Process


class PoolQueue(object):
    def __init__(self, n):
        self.num_procs = n

    def map(self, f, args):
        payloads = JoinableQueue()
        procs = []

        def add_task(arg):
            payloads.put(arg)

        def process_task():
            while True:
                pl = payloads.get()
                f(pl, add_task)
                payloads.task_done()

        for arg in args:
            add_task(arg)

        procs = [Process(target=process_task) for _ in range(self.num_procs)]
        for p in procs:
            p.start()

        payloads.join()
        for p in procs:
            p.kill()

这里的一个问题是，它死锁了队列增长到32767个任务。 from time import sleep from random import random def pause(): sleep(random() / 100) def process(payload, add_task): print(payload) pause() if payload: add_task(payload[:-1]) return payload if __name__ == '__main__': for x in range(1): PoolQueue(2).map( process, [ 'abcdefghij', '0123456789', '!@#$%^&*()', ], )可以更好地解决此问题，但这超出了此问题的范围。

用multiprocessing.Pool映射并同时附加到迭代器的Python方式是什么？

2 个答案: