将重试封装到`with`块中

时间:2013-06-04 13:46:36

标签: python exception-handling with-statement contextmanager

我希望将数据库事务的逻辑封装到with块中;将代码包装在事务中并处理各种异常(锁定问题)。这很简单,但是我想让块在某些异常后封装代码块的重试。我无法看到一种方法将它整齐地打包到上下文管理器中。

是否可以在with声明中重复代码?

我想像它一样使用它,这非常简洁。

def do_work():
    ...
    # This is ideal!
    with transaction(retries=3):
        # Atomic DB statements
        ...
    ...

我目前正在使用装饰器来处理这个问题,但是我更愿意提供上下文管理器(或实际上两者),所以我可以选择在with块中包含几行代码包含在装饰器中的内联函数,这是我现在所做的:

def do_work():
    ...
    # This is not ideal!
    @transaction(retries=3)
    def _perform_in_transaction():
        # Atomic DB statements
        ...
    _perform_in_transaction()
    ...

5 个答案:

答案 0 :(得分:13)

  

是否可以在with声明中重复代码?

No.

正如之前在邮件列表线程中指出的那样,您可以通过使装饰器调用传递的函数来减少一些重复:

def do_work():
    ...
    # This is not ideal!
    @transaction(retries=3)
    def _perform_in_transaction():
        # Atomic DB statements
        ...
    # called implicitly
    ...

答案 1 :(得分:6)

我这样做的方法只是实现标准数据库事务context manager,但允许它在构造函数中使用retries参数。然后我将在你的方法实现中将其包装起来。像这样:

class transaction(object):
    def __init__(self, retries=0):
        self.retries = retries
    def __enter__(self):
        return self
    def __exit__(self, exc_type, exc_val, traceback):
        pass

    # Implementation...
    def execute(self, query):
        err = None
        for _ in range(self.retries):
            try:
                return self._cursor.execute(query)
            except Exception as e:
                err = e # probably ought to save all errors, but hey
        raise err

with transaction(retries=3) as cursor:
    cursor.execute('BLAH')

答案 2 :(得分:4)

由于装饰器本身就是功能,您可以执行以下操作:

with transaction(_perform_in_transaction, retries=3) as _perf:
    _perf()

有关详细信息,您需要将transaction()实现为工厂方法,该方法返回设置为__callable__()的对象,以调用原始方法并将其重复到retries个失败的时候;对于数据库事务上下文管理器,__enter__()__exit__()将被定义为正常。

您也可以设置transaction(),使其自身执行传递的方法最多retries次,这可能需要与实现上下文管理器相同的工作量但是意味着实际使用量将减少到transaction(_perform_in_transaction, retries=3)(事实上,这相当于delnan提供的装饰器示例)。

答案 3 :(得分:1)

虽然我同意不能使用上下文管理器来完成...可以使用两个上下文管理器来完成!

结果有点尴尬,我不确定自己是否批准自己的代码,但这就是客户端的样子:

with RetryManager(retries=3) as rm:
    while rm:
        with rm.protect:
            print("Attempt #%d of %d" % (rm.attempt_count, rm.max_retries))
             # Atomic DB statements

仍然有一个明确的while循环,不是一个,而是两个with语句,这给我喜欢的错误留了太多机会。

代码如下:

class RetryManager(object):
    """ Context manager that counts attempts to run statements without
        exceptions being raised.
        - returns True when there should be more attempts
    """

    class _RetryProtector(object):
        """ Context manager that only raises exceptions if its parent
            RetryManager has given up."""
        def __init__(self, retry_manager):
            self._retry_manager = retry_manager

        def __enter__(self):
            self._retry_manager._note_try()
            return self

        def __exit__(self, exc_type, exc_val, traceback):
            if exc_type is None:
                self._retry_manager._note_success()
            else:
                # This would be a good place to implement sleep between
                # retries.
                pass

            # Suppress exception if the retry manager is still alive.
            return self._retry_manager.is_still_trying()

    def __init__(self, retries=1):

        self.max_retries = retries
        self.attempt_count = 0 # Note: 1-based.
        self._success = False

        self.protect = RetryManager._RetryProtector(self)

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, traceback):
        pass

    def _note_try(self):
        self.attempt_count += 1

    def _note_success(self):
        self._success = True

    def is_still_trying(self):
        return not self._success and self.attempt_count < self.max_retries

    def __bool__(self):
        return self.is_still_trying()

奖金:我知道您不想将您的工作分解为用装饰器包装的单独函数...但是,如果您对此感到满意,那么Mozilla的redo软件包提供装饰器来执行此操作,因此您不必自己动手。甚至还有一个Context Manager可以有效地充当您函数的临时装饰器,但是它仍然依赖于您的可检索代码将其分解为单个函数。

答案 4 :(得分:0)

这个问题已有几年历史了,但是在阅读了答案之后,我决定尝试一下。

此解决方案需要使用“帮助程序”类,但我认为它确实提供了一个接口,该接口具有通过上下文管理器配置的重试。

class Client:
    def _request(self):
        # do request stuff
        print("tried")
        raise Exception()

    def request(self):
        retry = getattr(self, "_retry", None)
        if not retry:
            return self._request()
        else:
            for n in range(retry.tries):
                try:
                    return self._request()
                except Exception:
                    retry.attempts += 1


class Retry:
    def __init__(self, client, tries=1):
        self.client = client
        self.tries = tries
        self.attempts = 0

    def __enter__(self):
        self.client._retry = self

    def __exit__(self, *exc):
        print(f"Tried {self.attempts} times")
        del self.client._retry


>>> client = Client()
>>> with Retry(client, tries=3):
    ... # will try 3 times
    ... response = client.request()

tried once
tried once
tried once
Tried 3 times