如何在Python中使用队列进行线程处理时处理异常

时间:2018-06-15 21:07:01

标签: python multithreading exception

我正在进行大量下载,可以从中提取任意数量的异常。我有代码似乎工作但不确定它会在所有情况下工作。

我有我的基本结构,它在重载的线程类中捕获错误,然后在该类中设置一个标志,并允许任何剩余的队列项循环而不执行任何操作。不应该中断已经运行的线程。

下面的结构中是否存在潜在的错误或是否可以改进?

之前提出的问题:How to handle exception in threading with queue in Python?

class DownloadWorker(Thread):
    my_error_message = None  # Critical Error Flag
    def __init__(self, queue):
        Thread.__init__(self)
        self.queue = queue

    def run(self):
        while True:
            thread_name = self.queue.get()
            if DownloadWorker.my_error_message:  
                self.queue.task_done()  # One by one, this will terminate remaining items in queue
            else:
                try:
                    self.dummy_download_data(thread_name)  # Process something here
                except Exception as e:
                    DownloadWorker.my_error_message = "Fatal Error in " + str(e)  # omitted lock/release
                finally:
                    self.queue.task_done()

    def dummy_download_data(self, thread_name):
        if thread_name == 'Thread2':  # mimic some error in some function way deep down
            raise Exception("? My Problem with " + thread_name)
        else:
            time.sleep(2)

main()
    thread_list = ['Thread1', 'Thread2', 'Thread3', 'Thread4', 'Thread5']
    queue = Queue()
    for x in range(3): 
        worker = DownloadWorker(queue)  
        worker.daemon = True  
        worker.start()  

    for name in thread_list: # Better way to handle exceptions thown in a thread?
        queue.put(name)

    queue.join()
    if DownloadWorker.my_error_message:  # Handling my error
        print(DownloadWorker.my_error_message)
    print('fini')

0 个答案:

没有答案