Python:使用多进程从外部脚本运行函数

时间:2017-08-23 15:52:59

标签: python python-2.7 concurrent.futures

我在使用concurrent.futures Multiprocess模块​​时遇到了一些问题。

import sys 
import ConfigParser
import os 
import concurrent.futures
from Script_A import get_list, AnalyzeRow # 

rec_list = get_rec_list(MODE)
with concurrent.futures.ProcessPoolExecutor(max_workers=4) as executor:
    for result in executor.map(AnalyzeRow, rec_list):
            print str(result) # This is just for testing

修改

我从Script_A导入的函数

def get_rec_list(mode):
    rowBank = []
    current_row = 0

    if mode == 'A':
       setup_logging()
    logging.debug("start")
    create_playlist()
    playlist_url= os.path.join(output_dir+"\\"+test_name, txtplaylist_name)
    if not os.path.exists(playlist_url):
        raise RuntimeError('playlist creation error')
    with open(playlist_url, 'rb') as input_ctrl_csv:
        ref_csv = csv.reader(input_ctrl_csv, delimiter=",")
        for row in ref_csv:
            if current_row > 0:
                row.append(current_row)
                rowBank.append(row)
            current_row += 1

    elif mode == 'B':
        setup_logging()
        logging.debug("start")
        playlist_url = os.path.join(sig_dir, sigplaylist_name)
        if not os.path.exists(playlist_url):
            raise RuntimeError('sig playlist error.')
        with open(playlist_url, 'rb') as sig_list:
            ref_csv = csv.reader(sig_list, delimiter=";")
            for row in ref_csv:
                if current_row > 0:
                    row.append(current_row)
                    rowBank.append(row)
                current_row += 1

    else: 
        raise RuntimeError('invalid run mode')

    return rowBank

def AnalyzeRow(row):
    #This function makes some math to check if the values extracted from the
    csv are inside a certain area range

    generateData(row)
    Error = getError(row)
    return Error

我知道Python会先前运行导入我当前代码的模块,但是当我在多进程函数中执行函数AnalyzeRow时,它会回想起我导入AnalyzeRow和get_rec_list的行。功能和运行之间的所有内容,但我没有得到任何结果。

编辑:当我运行我的脚本时,它会转到executor.map(AnalyzeRow, rec_list)行,然后跳回到行from Script_A import get_list, AnalyzeRow,这样它就会执行rec_list = get_rec_list(MODE) 3更多因为我使用4个处理器。当我执行我的代码时,我进入python交互式窗口3"启动"打印,对应get_rect_list()函数

有没有更好的方法可以执行此操作,或者我的代码中是否存在以错误方式调用多进程的错误?

0 个答案:

没有答案