Question

代码在一个数据集上工作，但落在下一个数据集上。

为解决Windows在多处理中的弱点，我专门从模块中导入了使用过的函数。最后一个数据集非常庞大（约180亿行）。

模块功能从scipy导入信号将numpy导入为np 将熊猫作为pd导入从alter_biosppy.signals导入eeg

def calculatestar(args):
    return calculate(*args)

def calculate(i,b,df,ind_dict):

   ids = ind_dict.get(b)

   df = df.loc[ids,:]

   Wn = 0.1
   k, l = signal.butter(3, Wn)

   for a in i:
       df.loc[:, "filtered_mine_"+a] = signal.filtfilt(k, l, df.loc[:, a]) 
   return   df

我如何运行多处理

if __name__ == '__main__':
    pool = multiprocessing.Pool()
    imap_it = pool.map(calculatestar, TASKS)
    pool.close()
    pool.join()

错误

OSError                                   Traceback (most recent call last)
<ipython-input-745-c3e94d82e56f> in <module>()
      3 if __name__ == '__main__':
      4     pool = multiprocessing.Pool()
----> 5     imap_it = pool.map(calculatestar, TASKS)
      6     pool.close()
      7     pool.join()

C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py in map(self, func, iterable, chunksize)
    266         in a list that is returned.
    267         '''
--> 268         return self._map_async(func, iterable, mapstar, chunksize).get()
    269 
    270     def starmap(self, func, iterable, chunksize=None):

C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py in get(self, timeout)
    655             return self._value
    656         else:
--> 657             raise self._value
    658 
    659     def _set(self, i, obj):

C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py in _handle_tasks(taskqueue, put, outqueue, pool, cache)
    429                         break
    430                     try:
--> 431                         put(task)
    432                     except Exception as e:
    433                         job, idx = task[:2]

C:\ProgramData\Anaconda3\lib\multiprocessing\connection.py in send(self, obj)
    204         self._check_closed()
    205         self._check_writable()
--> 206         self._send_bytes(_ForkingPickler.dumps(obj))
    207 
    208     def recv_bytes(self, maxlength=None):

C:\ProgramData\Anaconda3\lib\multiprocessing\connection.py in _send_bytes(self, buf)
    278 
    279         def _send_bytes(self, buf):
--> 280             ov, err = _winapi.WriteFile(self._handle, buf, overlapped=True)
    281             try:
    282                 if err == _winapi.ERROR_IO_PENDING:

OSError: [WinError 87]

如何修复“ [WinError 87]参数不正确”的Windows多处理程序？

0 个答案: