将对象添加到队列而不会中断

时间:2010-09-07 12:08:19

标签: python queue locking atomic

我想将两个对象放入队列中,但我必须确保对象同时在两个队列中,因此不应该在它们之间中断 - 就像原子块一样。有人有解决方案吗?非常感谢......

queue_01.put(car)
queue_02.put(bike)

2 个答案:

答案 0 :(得分:1)

您可以使用Condition object。您可以告诉线程等待cond.wait(),并在队列准备好cond.notify_all()时发出信号。例如,看看Doug Hellman的精彩Python Module of the Week blog。他的代码使用multiprocessing;在这里我已经为threading改编了它:

import threading
import Queue
import time

def stage_1(cond,q1,q2):
    """perform first stage of work, then notify stage_2 to continue"""
    with cond:
        q1.put('car')
        q2.put('bike')
        print 'stage_1 done and ready for stage 2'
        cond.notify_all()
def stage_2(cond,q):
    """wait for the condition telling us stage_1 is done"""
    name=threading.current_thread().name
    print 'Starting', name
    with cond:
        cond.wait()
        print '%s running' % name
def run():
    # http://www.doughellmann.com/PyMOTW/multiprocessing/communication.html#synchronizing-threads-with-a-condition-object
    condition=threading.Condition()
    queue_01=Queue.Queue()
    queue_02=Queue.Queue()    
    s1=threading.Thread(name='s1', target=stage_1, args=(condition,queue_01,queue_02))
    s2_clients=[
        threading.Thread(name='stage_2[1]', target=stage_2, args=(condition,queue_01)),
        threading.Thread(name='stage_2[2]', target=stage_2, args=(condition,queue_02)),
        ]
    # Notice stage2 processes are started before stage1 process, and yet they wait
    # until stage1 finishes
    for c in s2_clients:
        c.start()
        time.sleep(1)
    s1.start()
    s1.join()
    for c in s2_clients:
        c.join()

run()

运行脚本会产生

Starting stage_2[1]
Starting stage_2[2]
stage_1 done and ready for stage 2  <-- Notice that stage2 is prevented from running until the queues have been packed.
stage_2[2] running
stage_2[1] running

答案 1 :(得分:0)

要原子地添加到两个不同的队列,首先获取两个队列的锁。通过创建使用递归锁的Queue子类,这是最容易做到的。

import Queue # Note: module renamed to "queue" in Python 3
import threading

class MyQueue(Queue.Queue):
    "Make a queue that uses a recursive lock instead of a regular lock"
    def __init__(self):
        Queue.Queue.__init__(self)
        self.mutex = threading.RLock()

queue_01 = MyQueue()
queue_02 = MyQueue()

with queue_01.mutex:
    with queue_02.mutex:
        queue_01.put(1)
        queue_02.put(2)