I am using Python's DEAP pacakge and I want to multi-core my code and I used the tutorial at http://deap.gel.ulaval.ca/doc/dev/tutorials/distribution.html to successfully do it using multiprocessing.
My question is the following: using 8 cores, how much speedup to I get in theory? The reason I ask is because I want to decide how many individuals and generations I can run in the same amount of time as the single-cored version. My code used to take ~200s to run and with 8 cores, it now takes ~0.5 seconds (this is a 400X speedup). Can I assume that anything will be sped up by 400X? I know it's complex, but your help will be very appreciated.
In general, if anybody can help, I wanted to understand how multicoring changes the flow of computation. Does it just map the evaluation of each individual over different cores for each generation? Or does it run generations in parallel? If you know of any documentation I could read about this, please let me know.
I did not provide a code example as it seems not necessary because this is a very high-level question.
答案 0 :(得分：5)
它是否只是针对每一代在不同核心上映射每个人的评估，还是并行运行几代？
该示例映射evaluate操作，因此......
fitnesses = toolbox.map(toolbox.evaluate, invalid_ind)
单独的运行进程点击地图：所有invalid_ind被编组到一个队列中，当核心可用时，队列中的下一个人被分配到该核心运行evaluate例行程序。当队列为空时，所有结果将汇总到一个列表中并分配回fitnesses。因此，这个过程继续寂寞。
所以：
至少那是我I asked this question时所猜测的。根据您的应用程序，根据我对DEAP和cProfile的经验，CPU时间的前两位消费者正在评估个人和复制。