使用Queue()将命令发送到特定线程

时间:2015-06-24 15:21:14

标签: python multithreading message-queue

我想将消息发送到充当多个服务实例的控制器的特定线程。这是每个线程将用于检查适用于它的消息的代码段。评论应该解释它做得相当好(如果我的评论不好,请让我知道我应该怎么写它们!)

这不是一个错误修复问题 - 我是线程新手,想知道是否有更好的方法可以解决这个问题。对我来说感觉有点笨拙。

def check_queue(self):
      """Gets a simple lock on the command queue, checks its size, then
      iterates through jobs, looking for one that matches its uuid.
      If there is a match, break the out
      If not, put the job back on the queue, set job equal to None, and try again
  """
  with simple_lock:
      for i in range(cmd_q.qsize()):
          self.job = cmd_q.get()
          if job[0] == self.uuid
                break
          cmd_q.push(self.job)
          self.job = None
def command_check(self)
    while not self.job:
        sleep(.5) #wait for things to happen
        self.checkQueue()
    if self.job == 'kill':
        die_gracefully()
     ...etc
#example of a what commands are being passed
def kill_service(uuid):
    job = [uuid, 1]
    cmd_queue.put(job)
    #handle clean up

非常感谢任何帮助/评论/批评。

编辑:simple_lock只是threading.Lock()。我使用它来确保对cmd_q.qsize()的调用是准确的 - 否则,另一个进程可能会在调用qsize和执行作业之间添加到队列中。

2 个答案:

答案 0 :(得分:3)

你应该让每个线程都有自己独特的队列:创建一个dict,从u​​uid映射到与该uuid一起的Queue。生产者可以使用该dict来确定要推送到哪个队列,然后每个消费者只需使用q.get()而无需锁定,如果错误的uuid等可能重新排队。

我拿了你的示例代码并对其进行了扩展,以演示它的外观:

from Queue import Queue
import threading

class MyThread(threading.Thread):
    cmd_queues = {}  # Map of uuid -> Queue for all instances of MyThread

    def __init__(self, uuid):
        super(threading.Thread, self).__init__()
        self.cmd_q = Queue()
        self.uuid = uuid
        MyThread.cmd_queues[uuid] = self.cmd_q

    def run(self):
        while True:
            self.command_check()

    def command_check(self):
        job = self.cmd_q.get()
        if job == 'kill':
            self.die_gracefully()
         #...etc

    def die_gracefully(self):
       # stuff

def kill_service(uuid):
    job = 1
    MyThread.cmd_queues[uuid].put(job)
    #handle clean up

if __name__ == "__main__":
    for uuid in ['123', '234', '345']:
        t = MyThread(uuid)
        t.start()

    kill_service('123')

答案 1 :(得分:0)

使用Python 2.7,@ dano的答案(我觉得很棒)会引发一个RuntimeError - RuntimeError: thread.__init__() not called。我相信在调用super时,可以通过用基类名替换threading.Thread来解决异常。

from Queue import Queue
import threading

class MyThread(threading.Thread):
    cmd_queues = {}  # Map of uuid -> Queue for all instances of MyThread

    def __init__(self, uuid):
        super(MyThread, self).__init__()
        self.cmd_q = Queue()
        self.uuid = uuid
        MyThread.cmd_queues[uuid] = self.cmd_q

    def run(self):
        while True:
            self.command_check()

    def command_check(self):
        job = self.cmd_q.get()
        if job == 'kill':
            self.die_gracefully()
         #...etc

    def die_gracefully(self):
       # stuff
       pass

def kill_service(uuid):
    job = 1
    MyThread.cmd_queues[uuid].put(job)
    #handle clean up

if __name__ == "__main__":
    for uuid in ['123', '234', '345']:
        t = MyThread(uuid)
        t.start()

    kill_service('123')

如果这不是正确的解决方案,那么对此答案的编辑表示赞赏。