我遇到了这个问题,其中mp.Pool.join()在每次迭代中花费越来越多的时间。我无法在较小的测试用例上重新创建问题,但在此给出了实现的流程。任何见解将不胜感激。
while (ftime < etime):
results = []
pool = mp.Pool(10)
for tid in tracks:
results.append(pool.apply_async(processTrack, args=(tid, df, tracksH)))
pool.close()
pool.join()
for i in range(0, len(tracks)):
if (results[i].ready()):
gotresults = results[i].get()
insertrows = gotresults[0]
updaterows = gotresults[1]
ftime = ftime + oneminute
pool.join()调用花费的时间如下:
Iteration Time (s)
1 1.52
2 7.58
3 11.36
4 19.23
5 27.78
6 34.32
7 51.59