Python多处理池循环读写文本文件

时间:2018-01-29 11:49:10

标签: python parallel-processing multiprocessing pool joblib

我尝试使用多处理模块并行化循环以读取和写入大量文本文件。 这是我的问题: 我有一个名为SWAT的类,它包含几个方法。一种称为" auto"的方法尝试自动化所有其他方法。它需要3个args。 如何平行循环?

def auto (self, directory, number_samples, number_trajectories, grid_jump):

import glob, os
    os.chdir (directory)
    a = SWAT.generate_samples (self, number_samples, number_trajectories, 
grid_jump)
    b = [i.split (',') for i in SWAT.readlist (self, 'rng.txt')]

    for i in range (len(a)): 
        path = directory + '\\TxtInOut{}'. format (i)
        os.mkdir (path)
        sample = open ('sample.txt', 'w')
        for j in range (len (b)):
            sample.write ('{},{},{},{}\n'. format (b [j] [0], b [j] [1], b [j] [2], a [i] [j]))
        sample.close ()

        for file in glob.glob ('*cio'):
        e = SWAT.file_cio (self, file)
        c = SWAT.fig_fig (self, e [1] [6] [1])
        SWAT.wwq_write (self, SWAT.basins_bsn (self, e [1] [46] [1]) [1] [63] [1], path) 
        SWAT.basins_bsn_write (self, e [1] [46] [1], path)
        for j in c [0]:
            d = SWAT.subbasin (self, c [0] [j])
            SWAT.pnd_write (self, d [1] [30] [1], path)
            for k in d [3]:
                SWAT.hru_write (self, d [3] [k] [0], path)
                SWAT.mgt_write (self, d [3] [k] [1], path)
                SWAT.gw_write (self, d [3] [k] [4], path)
                SWAT.sep_write (self, d [3] [k] [6], path)
            SWAT.subbasin_write (self, c [0] [j], path)
        for j in c [1]:
            SWAT.rte_write (self, c [1] [j] [0], path)
            SWAT.swq_write (self, c [1] [j] [1], path)
            SWAT.file_cio_write (self, file, path)

0 个答案:

没有答案