Question

我正在运行以下代码：

import os
import logging
from multiprocessing import Pool

def clean_directories_mp(periodDays, dirs):
    chunks = [dirs[i::4] for i in range(4)]
    pool = Pool(processes=4)
    tup = [(chunks, periodDays)]
    pool.starmap(clean_directories, tup)
    return 0

def clean_directories(dirs, periodDays):
    print (dirs)
    print (periodDays)
    print (os.getpid())
    for directory in dirs:
        print (directory)
    return 0

dirs = get_directories(cleanupPath)
clean_directories_mp(periodDays, dirs)

不幸的是，它作为单个进程运行。如果我只使用pool.map而不是pool.starmap，它就可以了：

def clean_directories_mp(periodDays, dirs):
    chunks = [dirs[i::4] for i in range(4)]
    pool = Pool(processes=4)
    pool.map(clean_directories, chunks)
    return 0

def clean_directories(dirs, periodDays):
    print (dirs)
    print (periodDays)
    print (os.getpid())
    for directory in dirs:
        print (directory)
    return 0

dirs = get_directories(cleanupPath)
clean_directories_mp(periodDays, dirs)

我想将2个参数传递给clean_directories函数（dirs和periodDays列表的块）如何使用pool.starmap正确执行此操作？

Answer 1

chunks = [dirs[i::4] for i in range(4)]

块是列表

它看起来像： pool.map中，您可以传递一个“块”列表，它会针对每个进程自动进行迭代：

pool.map(clean_directories, chunks)

对于pool.starmap，您需要传递显式元组，因为不会自动迭代块列表：

for c in range(len(chunks)):
    arguments.append((chunks[c], periodDays))

使用pool.starmap代码时未作为多进程运行

1 个答案: