我在python中使用多重处理。这是我的代码。
def myfun(soup):
...
return param
if __name__ == '__main__':
res = requests.get('https://www.example.com', headers = headers, timeout = 1)
res.encoding = 'GB18030'
soup = BeautifulSoup(res.text, 'lxml')
with multiprocessing.Pool(4) as p:
val = p.map(myfun, [soup, soup])#same input for testing
但是它返回:
回溯(最近通话最近一次):
File "C:\main.py", line 1843, in <module>
val = p.map(myfun, [soup, soup])
File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 657, in get
raise self._value
File "C:\Program Files\Python37\lib\multiprocessing\pool.py", line 431, in _handle_tasks
put(task)
File "C:\Program Files\Python37\lib\multiprocessing\connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "C:\Program Files\Python37\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
RecursionError: maximum recursion depth exceeded while pickling an object
为什么会出现此错误?
如何解决?