Question

我有一个特殊的网址，我的代码暂停了几个小时（超过3个小时）。我似乎无法理解为什么会这样做。

网址为http://www.etudes.ccip.fr/maintenance_site.php。

直接的requests.get（）即时工作，但每当我有一个HTTPAdapter时，代码似乎几乎无限期地睡眠

import requests
from requests.adapters import HTTPAdapter    

url = 'http://www.etudes.ccip.fr/maintenance_site.php'
session = requests.Session()
session.mount('http://', HTTPAdapter(max_retries=2))
session.get(url, timeout=2)

Answer 1

您初始化的适配器使用下面的

设置重试

    if max_retries == DEFAULT_RETRIES:
        self.max_retries = Retry(0, read=False)
    else:
        self.max_retries = Retry.from_int(max_retries)

如果你看一下初始化

def __init__(self, total=10, connect=None, read=None, redirect=None, status=None,
             method_whitelist=DEFAULT_METHOD_WHITELIST, status_forcelist=None,
             backoff_factor=0, raise_on_redirect=True, raise_on_status=True,
             history=None, respect_retry_after_header=True):

respect_retry_after_header的默认值为True。你需要这个False。如果使用curl检查响应

$ curl -I http://www.etudes.ccip.fr/maintenance_site.php
HTTP/1.1 503 Service Temporarily Unavailable
Date: Thu, 23 Nov 2017 14:15:49 GMT
Server: Apache
Status: 503 Service Temporarily Unavailable
Retry-After: 3600
Expires: Sat, 26 Jul 1997 05:00:00 GMT
Cache-Control: pre-check=0, post-check=0, max-age=0
Pragma: no-cache
Connection: close
Content-Type: text/html; charset=ISO-8859-1

您希望respect_retry_after_header设置为False。这可以通过创建适配器然后修改此行为来完成

import requests
from requests.adapters import HTTPAdapter

url = 'http://www.etudes.ccip.fr/maintenance_site.php'
session = requests.Session()

adapter = HTTPAdapter(max_retries=2)
adapter.max_retries.respect_retry_after_header = False

session.mount('http://', adapter)

session.get(url, timeout=2)

Answer 2

响应中的retry-after标头是将连接设置为休眠3600秒的问题。请参阅urllib3中的retry.py。

    def sleep(self, response=None):
    """ Sleep between retry attempts.

    This method will respect a server's ``Retry-After`` response header
    and sleep the duration of the time requested. If that is not present, it
    will use an exponential backoff. By default, the backoff factor is 0 and
    this method will return immediately.
    """

    if response:
        slept = self.sleep_for_retry(response)
        if slept:
            return

    self._sleep_backoff()

解决方法是设置max_retries = 0。这样可以避免等待2x3600秒，直到您的申请结束。

Answer 3

正如Mylsal所说，尝试设置max_entries

session.mount('http://', HTTPAdapter(max_retries=0))

使用HTTPAdapter的Python请求暂停了几个小时

3 个答案: