Question

我正在玩多处理，特别是Pool。

import multiprocessing
import os
import time

results = []

def task(args):
    print "Start Process", os.getpid(), "with args", args, " timing", time.time(
)
    time.sleep(10)

    print "End   Process", os.getpid(), "with args", args, " timing", time.time(
)
    return os.getpid(), args, timestamp


def log_result(result):
   results.append(result)

if __name__ == "__main__":
   num_processes=64
   pool = multiprocessing.Pool(processes=num_processes)

   for i in range(num_processes):
       pool.apply_async(task, args=(i,), callback=log_result)

   pool.close()
   pool.join()

我的机器应该能够运行32个线程（根据/proc/cpuinfo文件，我有32个处理器。）

当我运行此脚本时，我得到以下结果：

Start Process 87166 with args 0  timing 1414054820.26
Start Process 87167 with args 1  timing 1414054820.26
Start Process 87168 with args 2  timing 1414054820.26
Start Process 87169 with args 3  timing 1414054820.26
Start Process 87170 with args 4  timing 1414054820.26
Start Process 87171 with args 5  timing 1414054820.26
Start Process 87172 with args 6  timing 1414054820.26
Start Process 87173 with args 7  timing 1414054820.26
Start Process 87174 with args 8  timing 1414054820.26
Start Process 87175 with args 9  timing 1414054820.26
Start Process 87176 with args 10  timing 1414054820.26
Start Process 87177 with args 11  timing 1414054820.26
Start Process 87178 with args 12  timing 1414054820.26
Start Process 87179 with args 13  timing 1414054820.26
Start Process 87180 with args 14  timing 1414054820.26
Start Process 87181 with args 15  timing 1414054820.26
Start Process 87182 with args 16  timing 1414054820.26
Start Process 87183 with args 17  timing 1414054820.26
Start Process 87184 with args 18  timing 1414054820.26
Start Process 87185 with args 19  timing 1414054820.26
Start Process 87186 with args 20  timing 1414054820.26
Start Process 87187 with args 21  timing 1414054820.26
Start Process 87188 with args 22  timing 1414054820.26
Start Process 87189 with args 23  timing 1414054820.26
Start Process 87190 with args 24  timing 1414054820.26
Start Process 87191 with args 25  timing 1414054820.26
Start Process 87192 with args 26  timing 1414054820.26
Start Process 87193 with args 27  timing 1414054820.26
Start Process 87194 with args 28  timing 1414054820.26
Start Process 87195 with args 29  timing 1414054820.26
Start Process 87196 with args 30  timing 1414054820.26
Start Process 87197 with args 31  timing 1414054820.26
Start Process 87198 with args 32  timing 1414054820.26
Start Process 87199 with args 33  timing 1414054820.26
Start Process 87200 with args 34  timing 1414054820.26
Start Process 87201 with args 35  timing 1414054820.26
Start Process 87202 with args 36  timing 1414054820.26
Start Process 87203 with args 37  timing 1414054820.26
Start Process 87204 with args 38  timing 1414054820.26
Start Process 87205 with args 39  timing 1414054820.26
Start Process 87206 with args 40  timing 1414054820.26
Start Process 87207 with args 41  timing 1414054820.26
Start Process 87208 with args 42  timing 1414054820.26
Start Process 87209 with args 43  timing 1414054820.26
Start Process 87210 with args 44  timing 1414054820.26
Start Process 87211 with args 45  timing 1414054820.26
Start Process 87212 with args 46  timing 1414054820.26
Start Process 87213 with args 47  timing 1414054820.26
Start Process 87214 with args 48  timing 1414054820.26
Start Process 87215 with args 49  timing 1414054820.26
Start Process 87216 with args 50  timing 1414054820.26
Start Process 87217 with args 51  timing 1414054820.26
Start Process 87218 with args 52  timing 1414054820.26
Start Process 87219 with args 53  timing 1414054820.26
Start Process 87220 with args 54  timing 1414054820.26
Start Process 87221 with args 55  timing 1414054820.26
Start Process 87222 with args 56  timing 1414054820.27
Start Process 87223 with args 57  timing 1414054820.27
Start Process 87224 with args 58  timing 1414054820.27
Start Process 87225 with args 59  timing 1414054820.27
Start Process 87226 with args 60  timing 1414054820.27
Start Process 87227 with args 61  timing 1414054820.27
Start Process 87228 with args 62  timing 1414054820.27
Start Process 87229 with args 63  timing 1414054820.27
End   Process 87166 with args 0  timing 1414054830.27
End   Process 87179 with args 13  timing 1414054830.27
End   Process 87229 with args 63  timing 1414054830.27
End   Process 87225 with args 59  timing 1414054830.27
End   Process 87223 with args 57  timing 1414054830.27
End   Process 87221 with args 55  timing 1414054830.27
End   Process 87219 with args 53  timing 1414054830.27
End   Process 87215 with args 49  timing 1414054830.27
End   Process 87191 with args 25  timing 1414054830.27
End   Process 87195 with args 29  timing 1414054830.27
End   Process 87171 with args 5  timing 1414054830.27
End   Process 87183 with args 17  timing 1414054830.27
End   Process 87189 with args 23  timing 1414054830.27
End   Process 87181 with args 15  timing 1414054830.27
End   Process 87185 with args 19  timing 1414054830.27
End   Process 87217 with args 51  timing 1414054830.27
End   Process 87175 with args 9  timing 1414054830.27
End   Process 87178 with args 12  timing 1414054830.27
End   Process 87213 with args 47  timing 1414054830.27
End   Process 87169 with args 3  timing 1414054830.27
End   Process 87193 with args 27  timing 1414054830.27
End   Process 87209 with args 43  timing 1414054830.27
End   Process 87177 with args 11  timing 1414054830.27
End   Process 87197 with args 31  timing 1414054830.27
End   Process 87187 with args 21  timing 1414054830.27
End   Process 87173 with args 7  timing 1414054830.27
End   Process 87172 with args 6  timing 1414054830.27
End   Process 87201 with args 35  timing 1414054830.27
End   Process 87176 with args 10  timing 1414054830.27
End   Process 87199 with args 33  timing 1414054830.27
End   Process 87205 with args 39  timing 1414054830.27
End   Process 87203 with args 37  timing 1414054830.27
End   Process 87207 with args 41  timing 1414054830.27
End   Process 87224 with args 58  timing 1414054830.27
End   Process 87208 with args 42  timing 1414054830.27
End   Process 87211 with args 45  timing 1414054830.27
End   Process 87212 with args 46  timing 1414054830.27
End   Process 87170 with args 4  timing 1414054830.27
End   Process 87167 with args 1  timing 1414054830.27
End   Process 87204 with args 38  timing 1414054830.27
End   Process 87227 with args 61  timing 1414054830.27
End   Process 87200 with args 34  timing 1414054830.27
End   Process 87216 with args 50  timing 1414054830.27
End   Process 87220 with args 54  timing 1414054830.27
End   Process 87196 with args 30  timing 1414054830.27
End   Process 87192 with args 26  timing 1414054830.27
End   Process 87184 with args 18  timing 1414054830.27
End   Process 87188 with args 22  timing 1414054830.27
End   Process 87180 with args 14  timing 1414054830.27
End   Process 87174 with args 8  timing 1414054830.27
End   Process 87168 with args 2  timing 1414054830.27
End   Process 87182 with args 16  timing 1414054830.27
End   Process 87186 with args 20  timing 1414054830.27
End   Process 87190 with args 24  timing 1414054830.27
End   Process 87202 with args 36  timing 1414054830.27
End   Process 87206 with args 40  timing 1414054830.27
End   Process 87210 with args 44  timing 1414054830.27
End   Process 87198 with args 32  timing 1414054830.27
End   Process 87194 with args 28  timing 1414054830.27
End   Process 87214 with args 48  timing 1414054830.27
End   Process 87228 with args 62  timing 1414054830.27
End   Process 87226 with args 60  timing 1414054830.27
End   Process 87222 with args 56  timing 1414054830.27
End   Process 87218 with args 52  timing 1414054830.27

所以，有两件事我真的不懂：

如何并行运行这64个进程？（因为所有开始时间和结束时间都相同）
Pool构造函数中processes的maxi值是多少？

Answer 1

答案1：您正在查看秒数，而不是毫秒数。提高准确性。

答案2：没有限制，但会为每个处理器分配processes/n_processors。

编辑：注意您的计算机上运行的进程超过32个，只需使用命令ps -A即可。这并不意味着您实际上并行运行，OS进程管理器正在为每个处理器轮换进程。有关详细信息，请参阅process management。

多处理池apply_async

1 个答案: