Question

我想使用扭曲的框架编写异步http客户端，它可以异步/同时向5个不同的服务器发出5个请求。然后比较这些响应并显示结果。有人可以帮忙解决这个问题。

Answer 1

对于这种情况，我建议使用treq和DeferredList来聚合响应，然后在返回所有网址时触发回调。这是一个简单的例子：

import treq
from twisted.internet import reactor, defer, task

def fetchURL(*urls):
    dList = []
    for url in urls:
        d = treq.get(url)
        d.addCallback(treq.content)
        dList.append(d)
    return defer.DeferredList(dList)

def compare(responses):
    # the responses are returned in a list of tuples
    # Ex: [(True, b'')]
    for status, content in responses:
        print(content)

def main(reactor):
    urls = [ 
        'http://swapi.co/api/films/schema',
        'http://swapi.co/api/people/schema',
        'http://swapi.co/api/planets/schema',
        'http://swapi.co/api/species/schema',
        'http://swapi.co/api/starships/schema',
        ]
    d = fetchURL(*urls)     # returns Deferred
    d.addCallback(compare)  # fire compare() once the URLs return w/ a response
    return d                # wait for the DeferredList to finish

task.react(main)
# usually you would run reactor.run() but react() takes care of that

在main函数中，网址列表会传递到fecthURL()。在那里，每个站点都会发出异步请求，并返回Deferred，并附加到list。然后最终列表将用于创建并返回DeferredList obj。最后，我们将compare()添加一个回调（DeferredList）来访问每个响应。您可以将比较逻辑放在compare()函数中。

Answer 2

您不一定需要twisted来发出异步http请求。您可以使用python线程和精彩的requests包。

from threading import Thread
import requests


def make_request(url, results):
    response = requests.get(url)
    results[url] = response


def main():
    results = {}
    threads = []
    for i in range(5):
        url = 'http://webpage/{}'.format(i)
        t = Thread(target=make_request, kwargs={'url': url, 'results': results})
        t.start()
        threads.append(t)

    for t in threads():
        t.join()

    print results

使用扭曲的框架编写异步http客户端

2 个答案: