我正在尽可能地简化我想要实现的目标 我有以下脚本:
import time, urllib, random
import threading
def get(timeout):
for i in range(2):
time.sleep(timeout)
return urllib.urlopen('http://localhost:9855').read()
def calculate_timeout(total_threads):
pass
if __name__ == '__main__':
total = random.randint(0,400)
for i in xrange(total):
threading.thread(target=get, kwargs={"timeout":calculate_timeout(len(total)}).start()
我需要做的是以这样的方式修改calculate_timeout
函数,以便随机编号出现,
平均每分钟不会超过60个urllib请求。
答案 0 :(得分:0)
发布了一系列限速算法here,其中包括线程代码的示例,最简单的' imo'我发布在下面。代码将创建一个装饰器,然后您可以使用它来装饰get
方法。
import time
import threading
from functools import wraps
def rate_limited(max_per_second):
"""
Decorator that make functions not be called faster than
"""
lock = threading.Lock()
min_interval = 1.0 / float(max_per_second)
def decorate(func):
last_time_called = [0.0]
@wraps(func)
def rate_limited_function(*args, **kwargs):
lock.acquire()
elapsed = time.clock() - last_time_called[0]
left_to_wait = min_interval - elapsed
if left_to_wait > 0:
time.sleep(left_to_wait)
lock.release()
ret = func(*args, **kwargs)
last_time_called[0] = time.clock()
return ret
return rate_limited_function
return decorate