Question

我的应用程序有一个运行时间长，CPU饥饿的线程（下面由Runner类示例），而线程池需要快速构建页面然后返回它（例如Builder类）。 Runner正在做背景媒体编码，并且对时间不敏感。 Builder.build_stuff()运行迅速且对时间敏感，因此如果尝试运行Runner，我想阻止Builder开始任何新工作。

我最初使用单个threading.Lock()来阻止Runner.do_work，但是限制了多个Builder一次不能运行。我在下面做了一个简化的示例，以显示我提出的策略；但我不是100％确信这是最好的解决方案，或者尚无现成的结构可以做到这一点。

import threading
runner = Runner()

"""Thread that is always running"""
def run():
    while True:
        runner.do_work()
running_work = threading.Thread(target=run).start()

"""Builders are created for a short duration by an outside pool of threads"""
class Builder:
    def __del__(self):
        runner.outside_release()

    def __init__(self):
        runner.outside_acquire()
        self.build_stuff()

    def build_stuff(self):
        """Do build some stuff here"""
        return """thing that was built"""

class Runner:
    def __init__(self):
        self.building_flag = 0
        self.building_lock = threading.Condition(threading.Lock())

    def outside_acquire(self):
        with runner.building_lock:
            runner.building_flag += 1
        self.build_stuff()

    def outside_release(self):
        with runner.building_lock:
            runner.building_flag -= 1
            runner.building_lock.notify()

    def do_work(self):
        with self.building_lock:
            while self.building_flag:
                self.building_lock.wait()
        """Do some work here"""

Answer 1

首先，您已经提到了这一点，但是我将对其他阅读此内容的人再说一遍：在长时间运行的线程中中断进程并重新启动它比将长时间运行的进程分成多个块并检查要困难得多。如果您应该经常停顿一下。

我发现您的示例有点混乱和不完整，因此我制作了一个单独的示例，说明如何解决上述问题。我使用了一个受threading.Lock保护的简单列表来跟踪所有正在运行的构建器，并使用threading.Event通知运行者何时停止和恢复工作。最重要的部分是构建器的run方法中“工作”之前和之后的部分，在该方法中，它自己将自身添加到正在运行的构建器列表中，并从中删除，并确定是否最后一次关闭了运行程序被通知重新开始。

from threading import Thread, Lock, Event
from time import sleep

class Builder(Thread):

    running = [] #list to keep track of any running builders
    running_lock = Lock() #mutex for running builder list

    def __init__(self, work, can_work_event):
        super().__init__()
        self.work = work
        self.runner_can_work = can_work_event


    def run(self):
        #before we do our work, add ourselves to running list and tell runner not to work
        self.runner_can_work.clear() #runner cannot start new work now
        with Builder.running_lock: #append and remove are not likely thread safe
            Builder.running.append(self) #add self to running builder list

        print(f'builder doing {self.work}')
        sleep(1)

        #this is not robust against builders crashing. Perhaps a better solution would
        #  keep track of thread id's, and periodically clean out id's of crashed threads
        #  from the Builder.running list

        #del isn't a reliable way ot determine when a task is done. Do this at the end
        #  of the work you intend to perform.

        with Builder.running_lock: #when builder is done with work
            Builder.running.remove(self) #remove self from list
            if not Builder.running: #no more builders are in the list
                self.runner_can_work.set() #allow runner to begin new work

class Runner(Thread):

    def __init__(self, work_to_do, can_work_event):
        super().__init__()
        self.work = work_to_do
        self.can_work = can_work_event

    def run(self):
        for chunk in self.work:
            self.can_work.wait() #wait on any running Builders
            print(f'runner working on {chunk}')
            sleep(2) #heavy computation

#example demonstration

runner_can_work = Event() #event used to notify runner of ability to work
runner_can_work.set() #set event to initially true (default is false)

r = Runner(range(10), runner_can_work) # range = dummy work to do
b1 = Builder('work 1', runner_can_work)
b2 = Builder('work 2', runner_can_work)
b3 = Builder('work 3', runner_can_work)
b4 = Builder('work 4', runner_can_work)
b5 = Builder('work 5', runner_can_work)

r.start()
sleep(3)
b1.start()
sleep(4)
b2.start()
b3.start()
sleep(3)
b4.start()
b5.start()

for t in (r,b1,b2,b3,b4,b5): t.join()
print('done')

当集合{B}的任何线程正在运行时，阻塞一个线程R

1 个答案: