如何将“with”与对象列表一起使用

时间:2017-09-07 00:44:59

标签: python multithreading

假设我有一个类将生成一个线程并实现.__enter__.__exit__,所以我可以这样使用它:

with SomeAsyncTask(params) as t:
    # do stuff with `t`
    t.thread.start()
    t.thread.join()

.__exit__可能会执行某些操作以进行清理(例如删除临时文件等)。

这一切都很好,直到我有一个SomeAsyncTask的列表,我想一次开始。

list_of_async_task_params = [params1, params2, ...]

我应该如何在列表中使用with?我希望有这样的事情:

with [SomeAsyncTask(params) for params in list_of_async_task_params] as tasks:
    # do stuff with `tasks`
    for task in tasks:
        task.thread.start()
    for task in tasks:
        task.thread.join()

3 个答案:

答案 0 :(得分:3)

我认为contextlib.ExitStack正是您正在寻找的。这是一种将不确定数量的上下文管理器安全地组合到一个上下文管理器中的方法(这样,当进入一个上下文管理器时,例外导致它跳过退出它已经成功输入的那个)

文档中的示例非常有启发性:

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

这可以适应您的希望"代码非常容易:

import contextlib

with contextlib.ExitStack() as stack:
    tasks = [stack.enter_context(SomeAsyncTask(params))
             for params in list_of_async_task_params]
    for task in tasks:
        task.thread.start()
    for task in tasks:
        task.thread.join()

答案 1 :(得分:2)

注意:不知怎的,我错过了你的Thread子类本身也是一个上下文管理器的事实 - 所以下面的代码没有做出这样的假设。然而,当使用更多“通用”类型的线程时(使用contextlib.ExitStack之类的东西不是一种选择)可能会有所帮助。

你的问题对细节有点了解 - 所以我做了一些 - 但这可能接近你想要的。它定义了一个AsyncTaskListContextManager类,它具有支持上下文管理器协议(以及相关的__enter__()语句)所需的必要__exit__()with方法。

import threading
from time import sleep

class SomeAsyncTask(threading.Thread):
    def __init__(self, name, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.name = name
        self.status_lock = threading.Lock()
        self.running = False

    def run(self):
        with self.status_lock:
            self.running = True

        while True:
            with self.status_lock:
                if not self.running:
                    break
            print('task {!r} running'.format(self.name))
            sleep(.1)

        print('task {!r} stopped'.format(self.name))

    def stop(self):
        with self.status_lock:
            self.running = False


class AsyncTaskListContextManager:
    def __init__(self, params_list):
        self.threads = [SomeAsyncTask(params) for params in params_list]

    def __enter__(self):
        for thread in self.threads:
            thread.start()
        return self

    def __exit__(self, *args):
        for thread in self.threads:
            if thread.is_alive():
                thread.stop()
                thread.join()  # wait for it to terminate
        return None  # allows exceptions to be processed normally

params = ['Fee', 'Fie', 'Foe']
with AsyncTaskListContextManager(params) as task_list:
    for _ in range(5):
        sleep(1)
    print('leaving task list context')

print('end-of-script')

输出:

task 'Fee' running
task 'Fie' running
task 'Foe' running
task 'Foe' running
task 'Fee' running
task 'Fie' running
... etc
task 'Fie' running
task 'Fee' running
task 'Foe' running
leaving task list context
task 'Foe' stopped
task 'Fie' stopped
task 'Fee' stopped
end-of-script

答案 2 :(得分:1)

@martineau答案应该有效。这是一个更通用的方法,应该适用于其他情况。请注意,__exit__()中不处理异常。如果一个__exit__()函数失败,则其他函数不会被调用。通用解决方案可能会抛出聚合异常并允许您处理它。另一个极端情况是,当您的第二个经理的__enter__()方法抛出异常时。在这种情况下,第一位经理的__exit__()将不会被调用。

class list_context_manager:
  def __init__(self, managers):
    this.managers = managers

  def __enter__(self):
    for m in self.managers:
      m.__enter__()
    return self.managers

  def __exit__(self):
    for m in self.managers:
      m.__exit__()

然后可以像你的问题一样使用它:

with list_context_manager([SomeAsyncTask(params) for params in list_of_async_task_params]) as tasks:
    # do stuff with `tasks`
    for task in tasks:
        task.thread.start()
    for task in tasks:
        task.thread.join()