我正在尝试使用pool.map()序列化一个函数,该函数接受一个输入并将输出写入csv文件。似乎该过程在第一组输入之后停止。我正在4个内核的Debian上运行代码。
if(oldnumofconversations != numofconversations)
{
m_Initialized = true;
oldnumofconversations = numofconversations;
}
from multiprocessing import Pool, cpu_count
from keras.models import load_model
input = [1,2,3,4,5,6,7,8]
model = load_model(directory)
def func(input):
# some calculations
prediction = model.predict(data)
# more calculations
print('Currently working on', input)
df.to_csv(directory)
def main():
workers = cpu_count()
pool = Pool(processes=workers)
pool.map(func, input)
pool.close()
pool.join()
if __name__ == '__main__':
start = time.time()
main()
print('That took {} seconds'.format(time.time() - start))
我得到上面的输出,然后就不继续进行后续输入了。我已经测试了该函数,没有进行多处理,而且效果很好,所以我猜问题出在Currently working on 1
Currently working on 2
Currently working on 3
Currently working on 4
中。
编辑: 经过一些调试后,我发现了问题。我正在使用Keras模型作为函数的一部分,这似乎是问题所在。