Question

我有一个python脚本，必须并行处理大量数据。但是每个作业都可以再次触发更多数据的加载。数据的所有处理都是独立的。这样我就可以通过使用并发来获得执行时间。

这是脚本的示例：

<form [formGroup]="ngMultiSelect">
<ng-multiselect-dropdown
  [placeholder]="'custom placeholder'"
  [data]="dropdownList"
  formControlName ="Name"
  [settings]="dropdownSettings"
  (onSelect)="onItemSelect($event)"
  (onSelectAll)="onSelectAll($event)"
  formControlName ="Name"
disabled="true"
>
</ng-multiselect-dropdown>
</form>

预期输出应为：

from concurrent.futures import ThreadPoolExecutor, as_completed, wait
from time import sleep


class RepoThreadPool(object):
    instance = None

    def __init__(self, thread_count=1):
        self.pool = ThreadPoolExecutor(max_workers=thread_count)
        self.futures = []
        self.results = []

        RepoThreadPool.instance = self

    def __del__(self):
        self.abort()

    def enqueue(self, func, *args, **kargs):
        future = self.pool.submit(func, *args, **kargs)
        self.futures.append(future)

    def join(self):
        for future in as_completed(self.futures):
        self.results.append(future.result())

    def abort(self, block=False):
        self.pool.shutdown(block)

def my_sleep(sleep_time):
    print('Start sleep %d' % sleep_time)
    sleep(sleep_time)
    print('Finished sleep %d' % sleep_time)
    return 1

def add_job():
    sleep(1)
    pool = RepoThreadPool.instance
    print('Add new job')
    pool.enqueue(my_sleep, 2)
    print('Done add new job')
    return 1

if __name__ == "__main__":
    pool = RepoThreadPool(2)
    print('Add job 1')
    pool.enqueue(my_sleep, 5)
    print('Add job 2')
    pool.enqueue(add_job)
    print('Add job 3')
    pool.enqueue(my_sleep, 3)

    print('Wait join')
    pool.join()
    print('Joined')

但这是

Add job 1
Start sleep 5
Add job 2
Add job 3
Wait join
Add new job
Done add new job
Start sleep 3
Finished sleep 3
Start sleep 2
Finished sleep 5
Finished sleep 2
Joined

因此，该联接不等待嵌套的提交作业。在此示例中出了什么问题？或者无法启动嵌套线程？

谢谢

具有递归的Python ThreadPoolExecutor无法正确加入

0 个答案: