Question

我有URI列表，列表中的每个URI应该并行命中多次。例如

uri_list = ["www.example1.com","www.example2.com", "www.example3.com"]
number_of_request = 10

在并行会话中，URL www.example1.com应该会命中10次。如何使用python请求来实现这一点。分享您的想法。

我试图这样做，但是没有用。

import threading
import time
import requests

start_time = time.time()
urls = ["http://www.google.com", "http://www.apple.com", "http://www.microsoft.com", "http://www.amazon.com", "http://www.facebook.com"]
num_of_req = 10
def fetch_url(url):
    print url
    start = time.time()
    for i in range(num_of_req):
        resp = requests.get(url)
    print "'%s\' fetched in %ss" % (url, (time.time() - start))

threads = [threading.Thread(target=fetch_url, args=(url,)) for url in urls]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

print "Elapsed Time: %s" % (time.time() - start_time)

所有uri都开始并行运行，但是每个uri的请求数都是按顺序进行的。

Answer 1

您可以简单地从for i in range(num_of_req)函数中删除fetch_url()并在创建urls数组的地方乘以threads数组：

threads = [threading.Thread(target=fetch_url, args=(url,)) for url in urls*num_of_req]

如何在python中并行从列表中多次击打每个uri？

1 个答案: