在项目列表上使用线程

时间:2019-09-10 15:05:39

标签: python multiprocessing

我有一个要使用线程处理的项目的列表,每个项目大约需要10分钟,我想一次做几个。

现在我一次只做一次,这需要一段时间。

old = ["old_item1","old_item2","old_item3","old_item4","old_item5"]
new = ["new_item1","new_item2","new_item3","new_item4","new_item5"]
area = ["area_item1","area_item2","area_item3","area_item4","area_item5"]

def func(x,y,z):
    print (x,y,z)

d = zip(old,new, area)

for e,f, g in d: 
  thread1 = threading.Thread(target=func, args=(e,f,g))
  thread2 = threading.Thread(target=func, args=(e,f,g))
  thread1.start()
  thread2.start()
  thread1.join()
  thread2.join()

所以我想做的是处理

带有线程1的item1 带有thread2的item2

带有线程1的项目3 带有thread2的item4

最后 线程1的item5

1 个答案:

答案 0 :(得分:1)

您可以使用multiprocessingpool,它将为您处理所有麻烦的工作。使用starmap函数,您可以将所有可迭代项作为参数运行一个函数。

类似的东西:

from multiprocessing import Pool

old = ["old_item1","old_item2","old_item3","old_item4","old_item5"]
new = ["new_item1","new_item2","new_item3","new_item4","new_item5"]
area = ["area_item1","area_item2","area_item3","area_item4","area_item5"]

def func(x,y,z):
    print (x,y,z)

d = zip(old,new, area)

with Pool() as pool:
    res = pool.starmap(func, d)
  • 这里实际上不需要保存结果,但是通常,返回值将是与给定迭代器相对应的所有结果的列表。