Question

我正在使用带有python3的请求模块来实现失败时的重试机制。

以下是我的代码 -

session = session or requests.Session()
retry = Retry(
    total=retries,
    read=retries,
    connect=retries,
    backoff_factor=backoff_factor,
    status_forcelist=status_forcelist,
    method_whitelist=method_whitelist)
retry.BACKOFF_MAX = 60
adapter = HTTPAdapter(max_retries=retry)
session.mount(BASE_URL, adapter)
return session

我没有看到任何命名参数来设置最大补偿（Retry类中的BACKOFF_MAX）。我不希望重试之间的休眠时间超过60秒。我怎样才能实现它？重置BACKOFF_MAX不起作用

Answer 1

我通过覆盖重试类

的get_backoff_time（）来实现这一点

class RetryRequest(Retry):

    DEFAULT_METHOD_WHITELIST = frozenset([
    'HEAD', 'GET', 'PUT', 'DELETE', 'OPTIONS', 'TRACE'])

    def __init__(self, total=10, connect=None, read=None, redirect=None, status=None,
             method_whitelist=DEFAULT_METHOD_WHITELIST, status_forcelist=None,
             backoff_factor=0, raise_on_redirect=True, raise_on_status=True,
             history=None, respect_retry_after_header=True, max_backoff=60):
        super(RetryRequest, self).__init__(total=total, connect=connect, read=read, redirect=redirect, status=status,
             method_whitelist=method_whitelist, status_forcelist=status_forcelist,
             backoff_factor=backoff_factor, raise_on_redirect=raise_on_redirect, raise_on_status=raise_on_status,
             history=history, respect_retry_after_header=respect_retry_after_header)
        self.backoff_max = max_backoff

    def get_backoff_time(self):
    """ Formula for computing the current backoff
    :rtype: float
    """
        # We want to consider only the last consecutive errors sequence (Ignore redirects).
        consecutive_errors_len = len(list(takewhile(lambda x: x.redirect_location is None,
                                                reversed(self.history))))
        if consecutive_errors_len <= 1:
            return 0

        backoff_value = self.backoff_factor * (2 ** (consecutive_errors_len - 1))
        return min(self.backoff_max, backoff_value)

并使用此类而不是重试

session = session or requests.Session()
retry = RetryRequest(
        total=retries,
        read=retries,
        connect=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
        method_whitelist=method_whitelist,
        max_backoff=max_backoff # in my case 60
)
adapter = HTTPAdapter(max_retries=retry)
session.mount(BASE_URL, adapter)
return session

Answer 2

我有同样的问题，这对我有用：

class RetryRequest(Retry):
    def __init__(self, backoff_max=60, **kwargs):
    super(RetryRequest, self).__init__(**kwargs)
    self.BACKOFF_MAX = backoff_max

我认为您不需要覆盖get_backoff_time()，只需覆盖构造函数并设置self.BACKOFF_MAX即可。此外，使用kwargs可以省略其他参数。

Answer 3

这是让我具有我想要的行为的

技巧：

from urllib3.util.retry import Retry

class RetryRequest(Retry):
    BACKOFF_MAX = 77

优点：

您不必重写任何重试方法（urllib3.util.retry）

缺点：

不是真实参数

其他答案似乎与“真实参数”情况有关。但是，您还必须重写new()方法：

from urllib3.util.retry import Retry

class RetryRequest(Retry):
    def __init__(self, backoff_max=Retry.BACKOFF_MAX, **kwargs):
        super().__init__(**kwargs)
        self.BACKOFF_MAX = backoff_max

    def new(self, **kwargs):
        return super().new(backoff_max=self.BACKOFF_MAX, **kwargs)

如果不覆盖它，它将使用__init__()定义中设置的默认参数。

不要犹豫，使用以下方法测试您的解决方案：

total=100
retry = RetryRequest(total=total, backoff_factor=0.1, backoff_max=77)

for i in range(0, total):
    print("Try #%d" % (total - retry.total + 1))
    retry = retry.increment()
    print(retry.get_backoff_time())

Answer 4

基于 https://stackoverflow.com/a/47688687/4833737 的函数覆盖思想，如果您希望它全局应用，则可以使用更简单的版本：

orig_backoff_fun = Retry.get_backoff_time

def custom_backoff_time(self):
    return min(5, orig_backoff_fun(self))

Retry.get_backoff_time = custom_backoff_time

用您想要的最长等待时间替换 5。

如何在处理请求重试时覆盖BACKOFF_MAX

4 个答案: