我正在尝试使用大量数据启动dask.cluster.Kmeans
。
因为我用numpy
包装dask.array
数组,所以可以使用CPU。
由于cupy
中未实现功能,因此似乎无法使用GPU。
我尝试重现Mattew Rocklin的示例(https://blog.dask.org/2019/01/03/dask-array-gpus-first-steps),该示例是从CuPy随机生成器生成随机dask数组的,虽然可以,但是我不想使用这种情况。
用cupy
包装dask.array
-不起作用。
>>> import dask.array as da
>>> import cupy as cp
>>> da.from_array(cp.arange(100000)).sum().compute()
我希望此数组的总和,但会出现以下错误:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/base.py", line 175, in compute
(result,) = compute(self, traverse=False, **kwargs)
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/base.py", line 446, in compute
results = schedule(dsk, keys, **kwargs)
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/threaded.py", line 82, in get
**kwargs
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/local.py", line 491, in get_async
raise_exception(exc, tb)
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/compatibility.py", line 130, in reraise
raise exc
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/local.py", line 233, in execute_task
result = _execute_task(task, data)
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/core.py", line 119, in _execute_task
return func(*args2)
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/dask/array/core.py", line 100, in getter
c = np.asarray(c)
File "/home/ubuntu/miniconda3/envs/cupy/lib/python3.6/site-packages/numpy/core/numeric.py", line 538, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: object __array__ method not producing an array
那么我如何通过dask数组管理CuPy的工作?
答案 0 :(得分:2)
从CuPy数组创建Dask数组时,需要提供da.from_array
关键字参数asarray=False
。因此,您的代码将如下所示。
>>> import dask.array as da
>>> import cupy as cp
>>> da.from_array(cp.arange(100000), asarray=False).sum().compute()