多处理池作业输出

时间:2017-06-14 21:43:18

标签: python python-2.7 multiprocessing jupyter-notebook

我正在使用开源项目来获得乐趣,但没有通过多处理池来使用BH-TSNE。

Github来源:

https://github.com/kylemcdonald/AudioNotebooks/blob/master/Fingerprints%20to%20t-SNE.ipynb

具体错误:

data = data.astype(np.float64)
def job(params):
    start = time()
    tsne(data, data_root, mode, initial_dims=params[0], perplexity=params[1])
    print 'initial_dims={}, perplexity={}, {} seconds'.format(params[0], params[1], time() - start)
params = list(itertools.product(initial_dims, perplexities))
pool = Pool()
pool.map(job, params)

utils/bhtsne.py:129: ComplexWarning: Casting complex values to real discards the imaginary part
  for sample in samples:
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-22-0350579d447b> in <module>()
      7 params = list(itertools.product(initial_dims, perplexities))
      8 pool = Pool()
----> 9 pool.map(job, params)

/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.pyc in map(self, func, iterable, chunksize)
    249         '''
    250         assert self._state == RUN
--> 251         return self.map_async(func, iterable, chunksize).get()
    252 
    253     def imap(self, func, iterable, chunksize=1):

/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.pyc in get(self, timeout)
    565             return self._value
    566         else:
--> 567             raise self._value
    568 
    569     def _set(self, i, obj):

AssertionError: ERROR: Call to bh_tsne exited with a non-zero return code exit status, please refer to the bh_tsne output for further details

我有bh_tsne包装器,二进制文件......

包装来源:https://github.com/lvdmaaten/bhtsne

但不清楚我是如何看待子进程输出的。

看了几个谷歌搜索,打开包装器中的详细模式为默认值..但是在DIR或Jupyter笔记本中似乎没有输出

1 个答案:

答案 0 :(得分:1)

似乎Jupyter Notebook在这类代码中表现不佳......但结果却是我的计算机环境问题..必须从源代码构建numpy并启用填充< / p>

https://github.com/numpy/numpy/issues/9254

重要的是..如果你在jupyter笔记本中崩溃,并且在游戏池中的一个线程中发生了粗鲁......没有办法看到跟踪AFAIK ..并且解决方法是直接在代码中运行代码笔记本或切换到非iPython