mpi4py传递dict对象

时间:2016-07-22 17:35:29

标签: python parallel-processing mpi4py

#mpiexec -n 3 python pass_dict.py
from mpi4py import MPI
import psycopg2
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

tax_dict={}
if rank == 0:
    tax_files=['2008','2009','2011','2012','2013','2014','2015']
    file_dir='/zhome/nah316/taxonomy/'
    for tax_file in tax_files:
        filename=file_dir + tax_file+'.csv'
        with open(filename,'r') as f:
            temp={}
            for line in f:
                temp_list=[]
                splitted_line = line.split()
                tag=splitted_line[1]
                temp_list.append(tag)
                temp[splitted_line[1]] = temp_list
            tax_dict[tax_file]=temp
else:
    tax_dict=None

comm.bcast(tax_dict, root = 0)

print  '-' * 20, rank , '-'* 30
print tax_dict['2015']['InvestmentSoldNotYetPurchasedRestrictedCost']    

这里是我尝试构建字典字典的代码,并将其传播到另外两个核心的通信器上。当我运行它时,我收到错误:

-------------------- 2 ------------------------------
Traceback (most recent call last):
  File "pass_dict.py", line 33, in <module>
    print tax_dict['2015']['InvestmentSoldNotYetPurchasedRestrictedCost']
TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------- 1 ------------------------------
Traceback (most recent call last):
  File "pass_dict.py", line 33, in <module>
    print tax_dict['2015']['InvestmentSoldNotYetPurchasedRestrictedCost']
TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------- 0 ------------------------------
['InvestmentSoldNotYetPurchasedRestrictedCost']

在我看来,传递给除root之外的核心的字典已经失去了作为字典的一些功能。为什么是这样?我应该如何绕过它并在根节点上进行传递?

提前致谢!

1 个答案:

答案 0 :(得分:1)

我没有详细了解python或mpi4py,但我在https://mpi4py.scipy.org/docs/usrman/tutorial.html发现了一些代码,这意味着你需要将comm.bcast的结果分配给其他等级的字典。

代码应为

tax_dict = comm.bcast(tax_dict, root = 0)

也许这解决了这个问题?