Question

我想更改Google Cloud Storage请求的默认超时和重试，并且可以看到api_core目录中的Google Cloud包中有装饰器函数，如何将它们应用到存储客户端（如果这是正确的术语）？

from google.cloud import storage
from google.api_core import timeout

timeout_ = timeout.ConstantTimeout(timeout=600)
storage_client = storage.Client()
storage_client.get_bucket(name_of_bucket)

Answer 1

在这里，您有一个基于api_core文件夹的自定义解决方案。区别在于func_with_timeout函数。通过这种方式，它可以为要包装的函数配置指定的超时和重试次数。

import functools
import signal
from multiprocessing import TimeoutError

import six
from google.cloud import storage


_PARTIAL_VALID_ASSIGNMENTS = ("__doc__",)


def wraps(wrapped):
    """A functools.wraps helper that handles partial objects on Python 2."""
    # https://github.com/google/pytype/issues/322
    if isinstance(wrapped, functools.partial):  # pytype: disable=wrong-arg-types
        return six.wraps(wrapped, assigned=_PARTIAL_VALID_ASSIGNMENTS)
    else:
        return six.wraps(wrapped)


def raise_timeout(signum, frame):
    raise TimeoutError


@six.python_2_unicode_compatible
class ConstantTimeout(object):
    """A decorator that adds a constant timeout argument.

    This is effectively equivalent to
    ``functools.partial(func, timeout=timeout)``.

    Args:
        timeout (Optional[float]): the timeout (in seconds) to applied to the
            wrapped function. If `None`, the target function is expected to
            never timeout.
    """

    def __init__(self, timeout=None, retries=None):
        self._timeout = timeout
        self._retries = retries

    def __call__(self, func):
        """Apply the timeout decorator.

        Args:
            func (Callable): The function to apply the timeout argument to.
                This function must accept a timeout keyword argument.

        Returns:
            Callable: The wrapped function.
        """

        @wraps(func)
        def func_with_timeout(*args, **kwargs):
            """Wrapped function that adds timeout."""
            signal.signal(signal.SIGALRM, raise_timeout)

            i = 1
            while i <= self._retries:
                try:
                    # Schedule the signal to be sent after ``time``.
                    signal.alarm(self._timeout)
                    return func(*args, **kwargs)

                except TimeoutError:
                    i += 1
                    pass

            return TimeoutError("Exceed maximum amount of retries: {}".format(self._retries))

        return func_with_timeout

    def __str__(self):
        return "<ConstantTimeout timeout={:.1f}>".format(self._timeout)

然后，您只需创建一个新函数，将要包装的函数作为参数传递给timeout_装饰器：

timeout_ = ConstantTimeout(timeout=2, retries=3)
storage_client = storage.Client()
get_bucket_with_timeout = timeout_(storage_client.get_bucket)
buck = get_bucket_with_timeout(name_of_bucket)
print(buck)

Answer 2

只需以秒为单位传递超时作为第二个参数：

storage_client.get_bucket(name_of_bucket,120)

如何在python中将超时和重试装饰器函数应用于google-cloud-storage客户端？

2 个答案: