Question

我正在使用＆＃34; requests-futures＆＃34;在异步get / post结果回调中打包和调用异步get / post（在futur结果上添加add_done_callback）。有时，我的代码会挂起。经过多个调查时间后，我可以使用最少的代码重现锁定：

from concurrent.futures import ThreadPoolExecutor
import time

pool = ThreadPoolExecutor(max_workers=10)

def f(_):
    time.sleep(0.1) # Try to force context switch
    x = pool.submit(lambda: None)
    print "1"
    x.result()
    print "2"

def main():
    x = pool.submit(lambda : None)
    x.add_done_callback(f)
    print "3"
    x.result()
    print "4"

print "==="
main()

如果我在bash循环中运行这段代码：

$> while true; do python code.py; done;

该程序每次都会使用＆＃34; trace＆＃34;：

挂起

(...)
===
1
2
3
4
===
3
4
1

如果我用ctrl ^ c打破它，我有以下堆栈跟踪：

^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/yienyien/Angus/test/futur/env/local/lib/python2.7/site-
packages/concurrent/futures/thread.py", line 46, in _python_exit
    t.join(sys.maxint)
  File "/usr/lib/python2.7/threading.py", line 951, in join
    self.__block.wait(delay)
  File "/usr/lib/python2.7/threading.py", line 359, in wait
    _sleep(delay)
KeyboardInterrupt
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/yienyien/Angus/test/futur/env/local/lib/python2.7/site-
packages/concurrent/futures/thread.py", line 46, in _python_exit
    t.join(sys.maxint)
 File "/usr/lib/python2.7/threading.py", line 951, in join
    self.__block.wait(delay)
  File "/usr/lib/python2.7/threading.py", line 359, in wait
    _sleep(delay)
KeyboardInterrupt

有人能解释我发生了什么吗？我在concurrent.futures模块中检查the possible deadlocks，但我认为它不匹配。

谢谢。

Answer 1

提交给固定大小的线程池的任务可能不会调用阻塞操作，如Future.result()。这会导致一种特定的死锁，称为“线程饥饿”。使用time.sleep()也可以从服务中切换一个线程，并增加线程饥饿的可能性。

Answer 2

我回答我自己的问题。经过调查，很简单。我没有关闭TheadPoolExecutor并且不使用with，有时main函数完成并且最终确定主线程，ThreadPoolExecutor状态变为＆＃34; shutdown＆＃34;而回调没有完成。

ThreadPoolExecutor在调用futur导致futur回调时挂起

2 个答案: