Question

我正在尝试将请求同时发送到服务器，然后使用以下代码记录平均延迟：

import Queue
import time
import threading
import urllib2

data = "{"image_1":"abc/xyz.jpg"}"
headers = {.....}
def get_url(q, url):
    num = 1
    sum = 0
    while num <= 200:
        start = time.time()
        req = urllib2.Request(url, data, headers)
        response = urllib2.urlopen(req)
        end = time.time()
        print end - start
        num = num + 1
        q.put(response.read())
        sum = sum + (end - start)
    print sum


theurls = ["http://example.com/example"]
q = Queue.Queue()

for u in theurls:
    t = threading.Thread(target = get_url, args = (q, u))
    t.daemon = True
    t.start()

while True:
    s = q.get()
    print s

此代码工作正常，但现在我打算每秒发送超过1000个请求。我遇到this answer，但我不确定如何将grequests用于我的案例。一些见解将非常有用。

由于

Answer 1

文档不是很好，但来源是。阅读来源！ Check out the first few lines of grequests.py on github：

"""
grequests
~~~~~~~~~
This module contains an asynchronous replica of ``requests.api``, powered
by gevent. All API methods return a ``Request`` instance (as opposed to
``Response``). A list of requests can be sent with ``map()``.
"""

The package exports the following：

__all__ = (
    'map', 'imap',
    'get', 'options', 'head', 'post', 'put', 'patch', 'delete', 'request'
)

Those symbols are defined further down the file：

# Shortcuts for creating AsyncRequest with appropriate HTTP method
get = partial(AsyncRequest, 'GET')
options = partial(AsyncRequest, 'OPTIONS')
head = partial(AsyncRequest, 'HEAD')
post = partial(AsyncRequest, 'POST')
put = partial(AsyncRequest, 'PUT')
patch = partial(AsyncRequest, 'PATCH')
delete = partial(AsyncRequest, 'DELETE')

partial was imported from functools at the top of the file.

from functools import partial

The documentation for functool.partial says:

返回一个新的部分对象，当调用它时，其行为类似于使用位置参数args和keyword arguments关键字调用的func。如果为调用提供了更多参数，则将它们附加到args。如果提供了其他关键字参数，它们会扩展和覆盖关键字。

基本上呼叫grequests.get来电AsyncRequest("GET", **other args go here**)。 AsyncRequest is a function which creates a new AsyncRequest.其文档说：

""" Asynchronous request.
Accept same parameters as ``Session.request`` and some additional:
:param session: Session which will do request
:param callback: Callback called on response.
                 Same as passing ``hooks={'response': callback}``
"""

会话已在前面定义：

from requests import Session

Here's a guide on using requests sessions.

在python中每秒发送超过1000个请求

1 个答案: