Question

我正在学习在网站某处（我添加print("check...")）中发现的多处理和运行代码，以观察使用apply_async运行的代码的顺序。下面是代码：

import multiprocessing
import time
def func(msg):
  for i in range(3):
    print(msg)
    time.sleep(1)
if __name__ == "__main__":
  pool = multiprocessing.Pool(processes=8)
  for i in range(10):
    msg = "hello %d" %(i)
    pool.apply_async(func, (msg, ))
    print('check whether block' + str(i))
  print('check')
  pool.close()
  pool.join()
  print("Sub-process(es) done.")

运行它时，得到以下输出：

hello 0
hello 2
hello 4
hello 1
hello 3
hello 5
hello 6
hello 7
check whether block0
check whether block1
check whether block2
check whether block3
check whether block4
check whether block5
check whether block6
check whether block7
check whether block8
check whether block9
check
hello 0
hello 4
hello 1
hello 5
hello 2
hello 3
hello 6
hello 7
hello 4
hello 0
hello 6
hello 3
hello 5
hello 2
hello 7
hello 1
hello 8
hello 9
hello 9
hello 8
hello 9
hello 8
Sub-process(es) done.

但是我希望看到的是以下内容：

hello 0
check whether block0
hello 2
check whether block2
hello 4
check whether block4
hello 1
check whether block1
hello 3
check whether block3
hello 5
check whether block5
hello 6
check whether block6
hello 7
check whether block7
hello 0
hello 4
hello 1
hello 5
hello 2
hello 3
hello 6
hello 7
hello 4
hello 0
hello 6
hello 3
hello 5
hello 2
hello 7
hello 1
hello 8
check whether block8
hello 9
check whether block9
hello 9
hello 8
hello 9
hello 8
check
Sub-process(es) done.

基本上我认为应该在每个print('check whether...')之后执行前8个pool.apply_async。释放2个步骤后，将打印8和9。并在最后打印支票。我所看到的似乎证明我错了，但我不知道代码序列的执行方式。

Answer 1

首先，请勿使用time.sleep进行类似的测试，因为python很聪明并且可能会做一些偷偷摸摸的事情。做类似sum(range(10_000_000))的操作，以确保python（和您的操作系统）没有将cpu用于其他操作。

第二，每次运行.apply_async()时，都会在不阻塞程序的情况下将一个任务添加到队列中，因此查看所有check ...是完全正常的，因为您实际上要检查的是是否将它们添加到队列中。

第三，启动一个过程要花一些时间，但是将任务添加到队列是瞬时的，这就是为什么在其余所有任务之前都看到所有check ...的原因。

我编辑了您的程序，使它更加明显。

def func(msg):
    for i in range(3):
        print(msg)
        sum(range(10_000_000))


if __name__ == "__main__":
    pool = multiprocessing.Pool(processes=8)
    for i in range(10):
        msg = f"hello {i: >{i+1}}"
        pool.apply_async(func, (msg, ))
        print('check whether block' + str(i))
    print('check')
    pool.close()
    pool.join()
    print("Sub-process(es) done.")

check whether block0             # <- adding task 0 to the queue
check whether block1             # <- adding task 1 to the queue
check whether block2             # <- adding task 2 to the queue
check whether block3             # <- adding task 3 to the queue
check whether block4             # <- adding task 4 to the queue
check whether block5             # <- adding task 5 to the queue
check whether block6             # <- adding task 6 to the queue
check whether block7             # <- adding task 7 to the queue
check whether block8             # <- adding task 8 to the queue
check whether block9             # <- adding task 9 to the queue
check                            #  ALL TASKS ADDED TO THE QUEUE
hello 0                          # -> Process 0 has been created to do task 0
hello  1                         # -> Process 1 has been created to do task 1
hello   2                        # -> Process 2 has been created to do task 2
hello    3                       # -> Process 3 has been created to do task 3
hello     4                      # -> Process 4 has been created to do task 4
hello      5                     # -> Process 5 has been created to do task 5
hello       6                    # -> Process 6 has been created to do task 6
hello        7                   # -> Process 7 has been created to do task 7
hello 0                          # -> Task is partly done
hello        7                   # -> Task is partly done
hello      5                     # -> Task is partly done
hello     4                      # -> Task is partly done
hello    3                       # -> Task is partly done
hello       6                    # -> Task is partly done
hello   2                        # -> Task is partly done
hello  1                         # -> Task is partly done
hello        7                   # -> Task is done
hello 0                          # -> Task is done
hello      5                     # -> Task is done
hello       6                    # -> Task is done
hello   2                        # -> Task is done
hello     4                      # -> Task is done
hello  1                         # -> Task is done
hello    3                       # -> Task is done
hello         8                  # -> Process 0 has been given task 8
hello          9                 # -> Process 7 has been given task 9
hello         8                  # -> Task is partly done
hello          9                 # -> Task is partly done
hello         8                  # -> Task is done
hello          9                 # -> Task is done
Sub-process(es) done.            # EVERYTHING DONE

在第一个Check whether block{X}之后保留hello {X+1}可能是由于您控制台输出不同进程和刷新的速度。

多进程pool.apply_async执行序列

1 个答案: