当使用urllib3.PoolManager(python 2.7,urllib3版本:1.21)时,我经常遇到ClosedPoolError异常。在阅读完代码之后,我倾向于认为这是预期的,因为返回的连接对象在使用时可能已被池的缓存逐出。
我已将错误复制到以下脚本中:
#!/bin/bash
set -ex
TEST_DIR=/tmp/urllib3_bug
mkdir -p $TEST_DIR
cd $TEST_DIR
virtualenv .virtualenv
. .virtualenv/bin/activate
pip install urllib3==1.21
# Starts 3 threads with pool size = 2
echo 'from urllib3 import PoolManager
import threading
from threading import Thread
pool_manager = PoolManager(num_pools=2)
def start(url):
print "Getting", url, "on", threading.current_thread().ident
return pool_manager.urlopen("GET", url)
thread1 = Thread(target=start, args=["http://localhost:9998"])
thread2 = Thread(target=start, args=["http://localhost:9999"])
thread3 = Thread(target=start, args=["http://localhost:10000"])
thread1.start()
import time
time.sleep(1)
thread2.start()
thread3.start()
' > start.py
# Simulates preemption by sleeping the thread
echo '303a304,308
> import time
> import threading
> print "Simulating preemption on", threading.current_thread().ident
> time.sleep(3)
> print "Waking up on", threading.current_thread().ident' | patch $TEST_DIR/.virtualenv/local/lib/python2.7/site-packages/urllib3/poolmanager.py -
rm -f $TEST_DIR/.virtualenv/local/lib/python2.7/site-packages/urllib3/poolmanager.pyc
python start.py
我创建一个大小为2的池,然后运行3个线程,每个线程连接到不同的URL。第一个线程应该遇到ClosedPoolError。
(补丁插入sleep()以在第一个线程上可靠地重现抢占。)