Multiprocess pool.starmap can't pickle arguments

时间:2018-07-25 05:24:03

标签: python multiprocessing closures

I'm having trouble running running a pool starmap command on a function shown in the code below. I receive the error AttributeError: Can't pickle local object 'Class_Name.Function_Outer.<locals>.Function_inner.<locals>.Function_Inner2'

Note it seems to be the starmap command itself causing the issue. I'm not sure if my inputs are wrong or what - normally it works for me.

Here is the code that seems to be causing the problem

from fabric import connection
import multiprocessing as mp

paths = ["/path/to/filename.txt", "/path/to/filename2.txt"]
pi_addresses = ["192.168.2.100", "192.168.2.101"]

...

def Inner_make_files(self, pi_addresses, paths):      

    #This list just creates a list of Raspberry Pis to SSH into with the fabric module
    pi_list = [connection.Connection(x, port=22, user="pi",
                connect_kwargs={"password" : “Raspberry”}) for x in self.pi_addresses]
    barrier = mp.Barrier(len(self.pi_list))

    def Inner2_create_file(self, pi, path):
        nonlocal barrier
        barrier.wait()
        pi.run(“raspistill -o {}”.format(path))  #Creates a new photo on the Pis
        pi.get(path, path) #Retrieves new file that was made

    print(paths)
    print(pi_list)
    pool = mp.Pool(processes = len(pi_list))
    pool.starmap(Inner2_create_file, zip(pi_list, paths))
    pool.close()
    pool.join()

The print statements output two lists, one of the photos and one of the connections. Both lists are two items long.

Any help would be much appreciated, thank you.

0 个答案:

没有答案