Question

我有一个python应用程序，它产生一个单独的进程来完成一些工作（由于GIL（全局解释器锁），我遇到了使用线程的性能问题）。现在我在python中使用什么方法来跨进程同步共享资源？

我将数据移动到一个队列中，并且当一个spawn进程从该队列接收数据时，它会完成工作。但我需要能够保证数据以有序的方式出现，与复制的顺序相同，所以我需要保证在任何时候只有一个进程可以从/向队列读/写。我该如何做到最好？

谢谢，罗恩

Answer 1

我认为你需要一个信号量，请查看这个示例代码：

import threading
import datetime


class ThreadClass(threading.Thread):
    def run(self):
        now = datetime.datetime.now()
        pool.acquire()
        print "%s says hello, World! at time: %s"  % (self.getName(),now)
        pool.release()


pool = threading.BoundedSemaphore(value=1)


for i in range(10):
        t = ThreadClass()
        t.start()

有这个输出：

Thread-1 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-2 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-3 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-4 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-5 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-6 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-7 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-8 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-9 says hello, World! at time: 2013-05-20 18:57:47.609000
Thread-10 says hello, World! at time: 2013-05-20 18:57:47.609000

其中：

import threading
import datetime


class ThreadClass(threading.Thread):
    def run(self):
        now = datetime.datetime.now()
        print "%s says hello, World! at time: %s"  % (self.getName(),now)




for i in range(10):
        t = ThreadClass()
        t.start()

有这个输出：

Thread-1 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-2 says hello, World! at time: 2013-05-20 18:58:05.
531000

 Thread-4 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-3 says hello, World! at time: 2013-05-20 18:58:05
.531000

 Thread-6 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-5 says hello, World! at time: 2013-05-20 18:58:05
.531000

 Thread-8 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-7 says hello, World! at time: 2013-05-20 18:58:05
.531000

 Thread-10 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-9 says hello, World! at time: 2013-05-20 18:58:0
5.531000

因此，在python中使用BoundedSemaphore，您可以确保在任何人写入队列之前，他们必须拥有信号量。但这并不能确保您的结果以正确的顺序添加到队列中。

编辑：

如果你打算这样做并保持秩序你需要这样的东西：

import multiprocessing
import datetime
import random
import time

def funfun(number):
    time.sleep(random.randint(0,10))
    now = datetime.datetime.now()
    return "%s says hello, World! at time: %s"  % (number,now)

if __name__ == "__main__":
    pool = multiprocessing.Pool(10)
    for item in pool.imap(funfun,[i for i in range(10)]):
        print item

打印：

0 says hello, World! at time: 2013-05-21 00:38:48.546000
1 says hello, World! at time: 2013-05-21 00:38:55.562000
2 says hello, World! at time: 2013-05-21 00:38:47.562000
3 says hello, World! at time: 2013-05-21 00:38:51.578000
4 says hello, World! at time: 2013-05-21 00:38:50.578000
5 says hello, World! at time: 2013-05-21 00:38:48.593000
6 says hello, World! at time: 2013-05-21 00:38:52.593000
7 says hello, World! at time: 2013-05-21 00:38:48.593000
8 says hello, World! at time: 2013-05-21 00:38:50.593000
9 says hello, World! at time: 2013-05-21 00:38:51.609000

因此，您可以按正确的顺序附加到队列，并且作业将等待轮到他们添加到队列中。

python中跨多个进程的同步

1 个答案: