Question

我发现Dask documentation非常混乱。假设我有一个函数：

import random
import dask

def my_function(arg1, arg2, arg3):
    val = random.uniform(arg1, arg2) 
    va2 = random.uniform(arg2, arg3)
    return val1 + val2

some_list = []
for i in range(100):
    some_num = dask.delayed(my_function)(arg1, arg2, arg3)
    some_list += [some_num]

computed_list = dask.compute(*some_list)

由于my_function()未获得全部3个参数，因此该操作将失败。

如何并行化dask中的这段代码？

编辑：

如果将 @dask.delayed 装饰器放在函数def顶部并正常调用，似乎可以正常工作，但是现在 .compute() < / strong>方法行引发：

KilledWorker: ('my_function-ac3c88f1-53f8-4d36-a520-ff8c40c6ee61', <Worker 'tcp://127.0.0.1:35925', name: 1, memory: 0, processing: 10>)

Answer 1

我先构建一个图，然后在其上调用计算：

import random
import dask

@dask.delayed
def my_function(arg1, arg2, arg3):
    val1 = random.uniform(arg1, arg2) 
    val2 = random.uniform(arg2, arg3)
    return val1 + val2

arg1 = 1
arg2 = 2
arg3 = 3

some_list = []
for i in range(10):
    some_num = my_function(arg1, arg2, arg3)
    some_list.append(some_num)

graph = dask.delayed()(some_list)
# graph.visualize()
computed_list = graph.compute()

如何使用Dask并行化循环？

1 个答案: