如何正常终止使用queue.Queue的多线程Python应用程序

时间:2018-07-04 12:45:16

标签: multithreading python-3.x queue terminate

很长一段时间以来,我一直在尝试让我的应用程序正常终止,但是到目前为止,我发现的答案都没有奏效。

下面的示例代码说明了我的应用程序的结构。它基本上是一串线程,这些线程使用队列将数据相互传递。

from abc import abstractmethod
from time import sleep
from threading import Thread, Event
from queue import Queue
import signal
import sys


class StoppableThread(Thread):

    def __init__(self):
        super().__init__()
        self.stopper = Event()
        self.queue = Queue()

    @abstractmethod
    def actual_job(self):
        pass

    def stop_running(self):
        self.stopper.set()


    def run(self):
        while not self.stopper.is_set():
            print(self.stopper.is_set())
            self.actual_job()
        self.queue.join()

class SomeObjectOne(StoppableThread):
    def __init__(self, name, some_object_two):
        super().__init__()
        self.name = name
        self.obj_two = some_object_two

    def actual_job(self):
        # print('{} is currently running'.format(self.name))
        input_string = 'some string'
        print('{} outputs {}'.format(self.name, input_string))
        self.obj_two.queue.put(input_string)
        sleep(2)

class SomeObjectTwo(StoppableThread):
    def __init__(self, name, some_object_three):
        super().__init__()
        self.name = name
        self.some_object_three = some_object_three


    def actual_job(self):
        # print('{} is currently running'.format(self.name))
        some_string = self.queue.get()
        inverted = some_string[::-1]
        print('{} outputs {}'.format(self.name , inverted))
        self.some_object_three.queue.put(inverted)
        sleep(2)


class SomeObjectThree(StoppableThread):
    def __init__(self, name):
        super().__init__()
        self.name = name

    def actual_job(self):
        print('{} is currently running'.format(self.name))
        some_string = self.queue.get()
        print('{} outputs {}'.format(self.name ,some_string[::-1]))
        sleep(2)




class ServiceExit(Exception):
    """
    Custom exception which is used to trigger the clean exit
    of all running threads and the main program.
    """
    pass

def service_shutdown(signum, frame):
    print('Caught signal %d' % signum)
    raise ServiceExit

signal.signal(signal.SIGTERM, service_shutdown)
signal.signal(signal.SIGINT, service_shutdown)

if __name__ == '__main__':
    thread_three = SomeObjectThree('SomeObjectThree')
    thread_two = SomeObjectTwo('SomeObjectTwo', thread_three)
    thread_one = SomeObjectOne('SomeObjectOne', thread_two)

    try:
        thread_three.start()
        thread_two.start()
        thread_one.start()

        # Keep the main thread running, otherwise signals are ignored.
        while True:
            sleep(0.5)

    except ServiceExit:
        print('Running service exit')
        thread_three.stop_running()
        thread_two.stop_running()
        thread_one.stop_running()
        thread_one.join()
        thread_two.join()
        thread_three.join()
        sys.exit(0)

现在,如果我运行此代码并按ctrl-C终止,则thread_one似乎已按预期加入,但是代码卡在了thread_two.join()上。

因为thread_one是唯一一个具有连续空队列的线程,所以我希望它与队列有关。

有什么想法吗?

1 个答案:

答案 0 :(得分:1)

run()的{​​{1}}方法中,您需要这样做:

StoppableThread

self.queue.join() is a blocking method

  

阻塞,直到队列中的所有项目都得到处理。

     

每当有项目添加到未完成任务中时,未完成任务的数量就会增加   队列。每当使用者线程调用时,计数就会减少   task_done()表示已检索到该项目及其所有工作   做完了。当未完成的任务数降至零时,join()   解除封锁。

因此,为了使join()返回,仅在另一个线程中join()的项目是不够的,还必须指出该项目已被get()处理:

task_done()