Question

我创建了一个通过ssh连接到多台机器的设置。这是我的配置文件：

c = get_config()

c.IPClusterEngines.engine_launcher_class = 'SSHEngineSetLauncher'
Clusters = [36,31,1,24,10,11,4,3,6,26,7,2,9]
c.SSHEngineSetLauncher.engines = dict( [ ('hostname%02d'%x,7) for x in Clusters ] )
c.SSHEngineSetLauncher.engine_args = ['--profile-dir=~/.ipython/profile_ssh']
c.LocalControllerLauncher.controller_args = ["--ip='*'"]

我有一个自定义类，并在下面得到错误。我无法理解的是，如果我连接到标准的ipcluster配置文件，我没有错误。为什么不同？

from IPython.parallel import Client
rc = Client() # standard
rcSSH = Client(profile='ssh') # SSH (this gives the error)
rc[:].use_dill()
rcSSH[:].use_dill()
rc[:].load_balanced_view().map_sync(customInstance.function, *args) # <- this runs fine
rcSSH[:].load_balanced_view().map_sync(customInstance.function, *args) # <- this gives the error

错误

ImportError                               Traceback (most recent call last)~/.local/lib/python2.7/site-packages/IPython/kernel/zmq/serialize.pyc in unpack_apply_message(bufs, g, copy)
    171     args = []
    172     for i in range(info['nargs']):
--> 173         arg, arg_bufs = unserialize_object(arg_bufs, g)
    174         args.append(arg)
    175     args = tuple(args)
~/.local/lib/python2.7/site-packages/IPython/kernel/zmq/serialize.pyc in unserialize_object(buffers, g)
    110         # a zmq message
    111         pobj = bytes(pobj)
--> 112     canned = pickle.loads(pobj)
    113     if istype(canned, sequence_types) and len(canned) < MAX_ITEMS:
    114         for c in canned:
~/.local/lib/python2.7/site-packages/dill/dill.pyc in loads(str)
    158     """unpickle an object from a string"""
    159     file = StringIO(str)
--> 160     return load(file)
    161 
    162 # def dumpzs(obj, protocol=None):
~/.local/lib/python2.7/site-packages/dill/dill.pyc in load(file)
    148     pik = Unpickler(file)
    149     pik._main_module = _main_module
--> 150     obj = pik.load()
    151     if type(obj).__module__ == _main_module.__name__: # point obj class to main
    152         try: obj.__class__ == getattr(pik._main_module, type(obj).__name__)
/usr/lib/python2.7/pickle.pyc in load(self)
    856             while 1:
    857                 key = read(1)
--> 858                 dispatch[key](self)
    859         except _Stop, stopinst:
    860             return stopinst.value
/usr/lib/python2.7/pickle.pyc in load_global(self)
   1088         module = self.readline()[:-1]
   1089         name = self.readline()[:-1]
-> 1090         klass = self.find_class(module, name)
   1091         self.append(klass)
   1092     dispatch[GLOBAL] = load_global
~/.local/lib/python2.7/site-packages/dill/dill.pyc in find_class(self, module, name)
    224         if (module, name) == ('__builtin__', '__main__'):
    225             return self._main_module.__dict__ #XXX: above set w/save_module_dict
--> 226         return StockUnpickler.find_class(self, module, name)
    227     pass
    228 
/usr/lib/python2.7/pickle.pyc in find_class(self, module, name)
   1122     def find_class(self, module, name):
   1123         # Subclasses may override this
-> 1124         __import__(module)
   1125         mod = sys.modules[module]
   1126         klass = getattr(mod, name)
ImportError: No module named fiberModes.GRINmediumArbPrec

修改

我应该提到，做以下事情并没有改变任何事情：

dview.execute('import fiberModes.GRINmediumArbPrec')

ipython并行wokrs超过默认设置但不超过ssh，为什么？

0 个答案: