Question

我正在使用Python urllib3 + requests的Retry模块来解决第三方API偶尔出现错误的情况。我遇到的一个问题是，如果重试持续失败，我将得到exceptions.MaxRetryError却从不知道响应是什么。如果服务器中有有价值的调试数据怎么办？

在抛出MaxRetryError的情况下，是否还有办法获得响应？

下面是我的代码

from requests.packages.urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter

def req_with_retry(retries=3, backoff_factor=0.5, status_forcelist=(400, 404, 500, 502, 504,), method_whitelist=frozenset(['POST', 'HEAD', 'TRACE', 'GET', 'PUT', 'OPTIONS', 'DELETE']), session=None,):
    ''' 
    this returns a session that functions like the requests module but with retries built it for certain status codes
    '''
    session = session or requests.Session()
    retry = Retry(
        total=retries,
        read=retries,
        connect=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
        method_whitelist=method_whitelist
    )
    adapter = HTTPAdapter(max_retries=retry)
    session.mount('http://', adapter)
    session.mount('https://', adapter)
    return session

r = req_with_retry().get(url="https://www.google.com")

Answer 1

首先，我想提一下，{url {3}不再被请求所取代，因此不再需要from requests.packages.urllib3.util.retry import Retry。考虑使用from urllib3.util.retry import Retry表单，因为它不涉及黑客行为（请参阅https://github.com/psf/requests/blob/v2.22.0/requests/packages.py）。

您无法同时获得MaxRetryError和urllib3的响应。但是您可以从urllib3获取响应，并在请求级别使用raise_for_status。但是，raise_for_status()不会受到status_forcelist的影响，因此您可能要自己重新实现它。

这是您的代码的改编，1 /重试状态2 /打印最终响应3 /引发异常。

import logging

import urllib3
import requests
import requests.adapters

logging.basicConfig(level=logging.DEBUG)
logging.getLogger("urllib3").setLevel(logging.DEBUG)


def req_with_retry(
    retries=3,
    backoff_factor=0.5,
    status_forcelist=(400, 404, 500, 502, 504),
    method_whitelist=frozenset(
        ["POST", "HEAD", "TRACE", "GET", "PUT", "OPTIONS", "DELETE"]
    ),
    session=None,
):
    """Returns a session that functions like the requests module but with retries
    built it for certain status codes
    """
    session = session or requests.Session()
    retry = urllib3.Retry(
        total=retries,
        read=retries,
        connect=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
        method_whitelist=method_whitelist,
        raise_on_status=False,
    )
    adapter = requests.adapters.HTTPAdapter(max_retries=retry)
    session.mount("http://", adapter)
    session.mount("https://", adapter)
    return session


r = req_with_retry().get(url="https://httpbin.org/status/404")
print(r.status_code, r.text)
r.raise_for_status()

urllib3：抛出MaxRetryError时如何获取响应？

1 个答案: