python for循环并行

时间:2016-04-18 12:34:32

标签: python loops parallel-processing

我正在尝试从输入文件中读取数据,并且每行都在while循环中执行任务。问题是当我创建第一个进程时 - 它的循环正在执行而不是将控制返回到上面的for循环。底线没有并行性。我做错了什么?

以下是相关代码:

from multiprocessing import Process


def work_line(list1Line,jobId):
    while True:
        print list1Line
        tenant = list1Line[0]
        module = list1Line[1]
        endTime = int(time.time())
        startTime = endTime - startTimeDelta
        generate(jobId, startTime, endTime, tenantServiceAddress, tenant, module)
        print ("tenant {} will sleep for {} seconds").format(tenant,sleepBetweenLoops)
        time.sleep(sleepBetweenLoops)


def openFiles():
    file = open(CLOUD_INPUT_FILE, 'r')
    lines = file.readlines()
    file.close()
    linesLen = len(lines)
    processes = []

    for linesIndex in range(0, linesLen):
        jobId = GenerateRandomID()
        line = lines[linesIndex]
        list1Line = line.split()

        p = Process(target=work_line(list1Line,jobId))
        p.start()
        processes.append(p)
        print processes

    for p in processes:
        p.join()


if __name__ == '__main__':
    CLOUD_INPUT_FILE = r'C:\CF\input_file.txt'
    tenantServiceAddress = 'address.address'
    startTimeDelta = 300
    sleepBetweenLoops = 1800
    print multiprocessing.cpu_count()
    openFiles()

1 个答案:

答案 0 :(得分:2)

你实际上在调用这个函数。改为

p = Process(target=work_line, args=(list1Line,jobId))