Question

在下面的程序中，我已经向队列发布了5个作业，但是只创建了3个线程。当我运行程序时，只完成了3个作业。我怎么能只用3个线程完成所有5个工作？有没有办法让一个已完成工作的线程接下一份工作？

import time
import Queue
import threading

class worker(threading.Thread):

    def __init__(self,qu):
        threading.Thread.__init__(self)
        self.que=qu

    def run(self):
        print "Going to sleep.."
        time.sleep(self.que.get())
        print "Slept .."
        self.que.task_done()


q = Queue.Queue()

for j in range(3):
    work = worker(q);
    work.setDaemon(True)
    work.start()


for i in range(5):
    q.put(1)


q.join()

print "done!!"

Answer 1

您需要让您的工作线程循环运行。您可以使用标记值（如None或自定义类）告诉工作人员在将所有实际工作项目放入队列后关闭：

import time
import Queue
import threading

class worker(threading.Thread):

    def __init__(self,qu):
        threading.Thread.__init__(self)
        self.que=qu

    def run(self):
        for item in iter(self.que.get, None): # This will call self.que.get() until None is returned, at which point the loop will break.
            print "Going to sleep.."
            time.sleep(item)
            print "Slept .."
            self.que.task_done()
        self.que.task_done()


q = Queue.Queue()

for j in range(3):
    work = worker(q);
    work.setDaemon(True)
    work.start()


for i in range(5):
    q.put(1)

for i in range(3):  # Shut down all the workers
    q.put(None)

q.join()

print "done!!"

另一种选择是使用multiprocessing.dummy.Pool，这是Python为您管理的线程池：

import time
from multiprocessing.dummy import Pool

def run(i):
    print "Going to sleep..."
    time.sleep(i)
    print "Slept .."

p = Pool(3)  # 3 threads in the pool

p.map(run, range(5)) # Calls run(i) for each element i in range(5)

p.close()
p.join()

print "done!!"

如何在Python中使用固定数量的线程处理队列中的多个作业

1 个答案: