Question

我有以下代码：

while True:
    try:
        for i in range(2):
            t1 = Thread(target = grab_data_from_queue)
            t1.daemon = True
            t1.start() # start the thread
    except (KeyboardInterrupt, SystemExit):
      print '\n[!] Received keyboard interrupt, quitting threads.\n'
      exit()

正在运行的函数正在等待将项添加到队列中 - 我希望这是一个长期运行的任务，因此该函数可以处理和处理任务。

但是，这会不断产生新的线程，我不想让它在一夜之间运行，并且有成千上万的处理线程。

更新：可能的解决方法是根据评论执行以下操作：

while True:
    if q.empty():
        time.sleep(5)
    else:
        try:
            for i in range(2):
                t1 = Thread(target = grab_data_from_queue)
                t1.daemon = True
                t1.start() # start the thread
        except (KeyboardInterrupt, SystemExit):
          print '\n[!] Received keyboard interrupt, quitting threads.\n'
          exit()

Answer 1

如果grab_data_from_queue func是一个cpu密集型工作，您可以尝试使用concurrent.futures.ProcessPoolExecutor()，那么您的代码可能是这样的：

pool = concurrent.futures.ProcessPoolExecutor(max_workers=3)
while True:
    if q.empty():
        time.sleep(5)
    else:
        futures = set()
        with pool as executor:
            future = executor.submit(grab_data_from_queue, data)
            futures.add(future)
        try:
            for future in concurrent.futures.as_completed(futures):
                exc = future.exception()
                if exc:
                    raise exc

        except (KeyboardInterrupt, SystemExit):
          print '\n[!] Received keyboard interrupt, quitting threads.\n'
          exit()

Python线程 - 这会不会产生新的线程？

1 个答案: