从Dask中的任务启动任务

Question

我必须使用dask从任务中启动任务。尽管此处对此进行了很好的解释：https://distributed.readthedocs.io/en/latest/task-launch.html#launch-tasks-from-tasks，但我无法使其正常运行。启动调度程序后

dask-scheduler --scheduler-file /tmp/scheduler.json

和一个工人

dask-worker --scheduler-file /tmp/scheduler.json

以下代码

import numpy as np
from dask.distributed import Client, secede, rejoin, get_client
from traceback import print_exc

def is_prime(n):

    for i in range(2,n):
        if (n % i) == 0:
            return False
    return True        

def recursive_function(depth, data, max_n, scheduler_file=None):

    futures = []
    non_primes = []
    try:
        if depth > 3:
            return []
        else:
            client = None
            if scheduler_file is not None:
                client = Client(scheduler_file=scheduler_file, processes=False)
                secede()
            for i in data:
                if is_prime(i):
                    new_data = np.random.randint(2, max_n, len(data))
                    if client is not None:
                        future = client.submit(recursive_function, depth+1,
                                new_data, max_n, scheduler_file)
                        futures.append(future)
                    else:
                        new_numbers = recursive_function(depth+1, new_data,
                                max_n, scheduler_file)
                        non_primes.append(new_numbers)
                else:
                    non_primes.append(i)

            if futures:
                print('N FUTURES: {}'.format(len(futures)))
                non_primes.append(client.gather(futures))
                rejoin()
    except:
        print_exc()
        if client:
            client.close()

    return non_primes

产生

In [1]: from tester import recursive_function                                                                                                                                                           
   ...: max_n = 100                                                                                                                                                                                     
   ...: data = np.random.randint(2, max_n, 100)                                                                                                                                                         
   ...: %time r_dask = recursive_function(0, data, max_n, scheduler_file='/tmp/scheduler.json')                                                                                                         
   ...:                                                                                                                                                                                                 
Traceback (most recent call last):                                                                                                                                                                      
  File "/usr/local/lib/python3.5/dist-packages/distributed/worker.py", line 2590, in get_worker                                                                                                         
    return thread_state.execution_state['worker']                                                                                                                                                       
AttributeError: '_thread._local' object has no attribute 'execution_state'                                                                                                                              

During handling of the above exception, another exception occurred:                                                                                                                                     

Traceback (most recent call last):                                                                                                                                                                      
  File "/home/ppazzini/Documentos/projetos/spots/tester.py", line 42, in recursive_function                                                                                                             
    secede()                                                                                                                                                                                            
  File "/usr/local/lib/python3.5/dist-packages/distributed/worker.py", line 2664, in secede                                                                                                             
    worker = get_worker()                                                                                                                                                                               
  File "/usr/local/lib/python3.5/dist-packages/distributed/worker.py", line 2596, in get_worker                                                                                                         
    raise ValueError("No workers found")                                                                                                                                                                
ValueError: No workers found                                                                                                                                                                            
CPU times: user 50.5 ms, sys: 2.5 ms, total: 53 ms                                                                                                                                                      
Wall time: 53.9 ms

我的代码或逻辑中是否有错误？如果没有，有可能达到我想要的目标吗？