Question

我正在编写一个脚本来并行查询多个vcenter。我已经有一个脚本以循环方式执行此操作，但是它确实很慢，因为与每个vcenter的连接是一个接一个地建立的。需要几分钟才能完成，在某些情况下这是不可接受的。

所以，我的想法是并行化返回会话的函数，然后将这些会话存储在数组或字典中，以使用它们查询vecenter。

这是代码

from pyVmomi import vim
import threading
from multiprocessing import Process
from multiprocessing import Manager

serverList = ["vcenter1", "vcenter2", "vcenter3", "vcenter4",
              "vcenter5", "vcenter6", "vcenter7", "vcenter8",
              "vcenter9", "vcenter10", "vcenter11", "vcenter12",
              "vcenter13", "vcenter14", "vcenter15", "vcenter16",
              "vcenter17", "vcenter18"]

def createVcenterSession(sessions, vcenter="", user="", passwd="", ssl=False):
    url= vcenter + "mydomain.com"

    if not ssl:
        from pyVim.connect import SmartConnectNoSSL
        service_instance = SmartConnectNoSSL(host=url,
                                             user=user,
                                             pwd=passwd,
                                             port=int(443))
    else:
        from pyVim.connect import SmartConnect
        service_instance = SmartConnect(host=url,
                                        user=user,
                                        pwd=passwd,
                                        port=int(443))
if __name__ == '__main__':

    manager = Manager()
    global sessions_dict
    sessions_dict = manager.dict()

    d = [] ##Here, I will Store the processes
    for a in serverList:
        d.append(Process(target=createVcenterSession, args=(sessions_dict, a, "USERNAME","PASSWD", False)))
    ### now d is filled with all the new processes

    ####start everyone
    for a in d:
        a.start()

    print (d)
    ####wait for everyone to complete
    for a in d:
        a.join()

    print (sessions_dict) ##Show my sessions

问题是代码会引发此错误

Process Process-16:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "dashBoard.py", line 54, in createVcenterSession
    sessions[vcenter] = service_instance
  File "<string>", line 2, in __setitem__
  File "/usr/lib/python3.5/multiprocessing/managers.py", line 716, in _callmethod
    conn.send((self._id, methodname, args, kwds))
  File "/usr/lib/python3.5/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/usr/lib/python3.5/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'pyVmomi.VmomiSupport.vim.ServiceInstance'>: attribute lookup vim.ServiceInstance on pyVmomi.VmomiSupport failed

我知道问题在于Multiprocess lib能够将ServiceInstance类放入托管字典中。如果更改从以下位置分配字典的行

    sessions[vcenter] = service_instance

到

    sessions[vcenter] = "any string"

它的工作原理很吸引人，但是我却失去了会话：）

有什么想法吗？

预先感谢

Zeke

多进程和pyVmomi引发_pickle.PicklingError

0 个答案: