Question

你好我想多处理for循环，我计算4个不同的独立函数。唯一的问题是这些函数需要多个参数。其中一些参数是我使用循环索引选择正确元素的列表。这是一个示例代码

import numpy as np

def function_1(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
def function_2(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
def function_3(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
def function_4(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
result_1 = 0
result_2 = 0
result_3 = 0
result_4 = 0
a = [1,2,3]
b = [4,5,6]
c = 2
d = 5
for k in xrange(3):
    result_1 += function_1(a[k],b[k],k*c,d)
    result_2 += function_2(a[k],b[k],k*c,d)
    result_3 += function_3(a[k],b[k],k*c,d)
    result_4 += function_4(a[k],b[k],k*c,d)

有没有办法并行而不是顺序进行这些计算？是否可以为用fortran编写的函数执行此操作？

Answer 1

def gen_func(idx, *args):
  def function_1(*args):
    i_want_2 = args[:2]
    return sum(i_want_2)
  def function_2(*args):
    i_want_3 = args[:3]
    return sum(i_want_3)
  def function_3(*args):
    return sum(args)
  def function_4(*args):
    return sum(args)

  result = 0 
  func = {0:function_1,1:function_2,2:function_3,3:function_4}
  for a_k, b_k in zip(args[0], args[1]):
    result += func[idx](a_k, b_k,*args[2:])

  print idx, result




import multiprocessing
  jobs = []
  a = [1,2,3]
  b = [4,5,6]
  c = 2
  d = 5
  for i in range(4):
    p = multiprocessing.Process(target=gen_func, args=(i,a,b,c,d,))
    jobs.append(p)
    p.start()

输出：

Answer 2

我只是在考虑做以下事情会不会更容易：

import numpy as np

def function_1(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
def function_2(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
def function_3(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
def function_4(a,b,c,d):
    #do heavy calculations
    #function is written in fortran and linked via f2py
result_1 = 0
result_2 = 0
result_3 = 0
result_4 = 0
a = [1,2,3]
b = [4,5,6] 
c = 2
d = 5
for k in xrange(3):
    p_1 = multiprocessing.Process(target = function_1, (a[k],b[k],k*c,d))
    p_1.start()
    p_2 = multiprocessing.Process(target = function_2, (a[k],b[k],k*c,d))
    p_2.start()
    p_3 = multiprocessing.Process(target = function_3, (a[k],b[k],k*c,d))
    p_3.start()
    p_4 = multiprocessing.Process(target = function_4, (a[k],b[k],k*c,d))
    p_4.start()
    #wait till process finishes is there such an option?
    result_1 += p_1
    result_2 += p_2
    result_3 += p_3
    result_4 += p_4
    #does the process return a complex number?

使用依赖于循环索引的多个参数对for循环进行多处理

2 个答案: