Question

今天有趣的用例：我需要在代码更改后迁移代码库中的模块。旧mynamespace.Document将消失，我希望通过代码对象替换此包来确保顺利迁移，该代码对象将动态导入正确的路径并迁移相应的对象。

简而言之：

# instanciate a dynamic package, but do not load
# statically submodules
mynamespace.Document = SomeObject()
assert 'submodule' not in mynamespace.Document.__dict__

# and later on, when importing it, the submodule
# is built if not already available in __dict__
from namespace.Document.submodule import klass
c = klass()

有几点需要注意：

我不只是谈论迁移代码。从某种意义上说，一个简单的巨大sed足以改变代码以迁移一些导入，我不需要动态模块。我在谈论对象。一个拥有一些实时/存储对象的网站需要迁移。假设mynamespace.Document.submodule.klass存在，将加载这些对象，这就是动态模块的原因。我需要为网站提供一些内容。
我们不能或不想改变对象被取消/加载的方式。为简单起见，我们只想说我们要确保成语from mynamespace.Document.submodule import klass必须起作用。我无法使用from mynamespace import Document as container; klass = getattr(getattr(container, 'submodule'), 'klass')

我尝试了什么：

import sys
from types import ModuleType

class VerboseModule(ModuleType):
    def __init__(self, name, doc=None):
        super(VerboseModule, self).__init__(name, doc)
        sys.modules[name] = self
    def __repr__(self):
        return "<%s %s>" % (self.__class__.__name__, self.__name__)
    def __getattribute__(self, name):
        if name not in ('__name__', '__repr__', '__class__'):
            print "fetching attribute %s for %s" % (name, self)
        return super(VerboseModule, self).__getattribute__(name)

class DynamicModule(VerboseModule):
    """
    This module generates a dummy class when asked for a component
    """
    def __getattr__(self, name):
        class Dummy(object):
            pass
        Dummy.__name__ = name
        Dummy.__module__ = self
        setattr(self, name, Dummy)
        return Dummy
class DynamicPackage(VerboseModule):
    """
    This package should generate dummy modules
    """
    def __getattr__(self, name):
        mod = DynamicModule("%s.%s" % (self.__name__, name))
        setattr(self, name, mod)
        return mod

DynamicModule("foobar")
# (the import prints:)
# fetching attribute __path__ for <DynamicModule foobar>
# fetching attribute DynamicModuleWorks for <DynamicModule foobar>
# fetching attribute DynamicModuleWorks for <DynamicModule foobar>
from foobar import DynamicModuleWorks
print DynamicModuleWorks

DynamicPackage('document')
# fetching attribute __path__ for <DynamicPackage document>
from document.submodule import ButDynamicPackageDoesNotWork
# Traceback (most recent call last):
# File "dynamicmodule.py", line 40, in <module>
#   from document.submodule import ButDynamicPackageDoesNotWork
#ImportError: No module named submodule

正如您所看到的，动态包不起作用。我不明白发生了什么，因为document甚至没有要求ButDynamicPackageDoesNotWork属性。

任何人都可以澄清正在发生的事情;以及如何/如何解决这个问题？

Answer 1

问题是python将绕过document中sys.modules的条目并直接加载submodule的文件。当然这不存在。

示范：

>>> import multiprocessing
>>> multiprocessing.heap = None
>>> import multiprocessing.heap
>>> multiprocessing.heap
<module 'multiprocessing.heap' from '/usr/lib/python2.6/multiprocessing/heap.pyc'>

我们希望heap仍然是None，因为python可以将它从sys.modules中拉出来，但这不会发生。点缀符号本质上直接映射到{something on python path}/document/submodule.py，并尝试直接加载它。

更新

诀窍是覆盖pythons导入系统。以下代码需要您的DynamicModule类。

import sys

class DynamicImporter(object):
    """this class works as both a finder and a loader."""
    def __init__(self, lazy_packages):
        self.packages = lazy_packages

    def load_module(self, fullname):
        """this makes the class a loader. It is given name of a module and expected
           to return the module object"""
        print "loading {0}".format(fullname)
        components = fullname.split('.')
        components = ['.'.join(components[:i+1])
                      for i in range(len(components))]
        for component in components:
            if component not in sys.modules:
                DynamicModule(component)
                print "{0} created".format(component)
        return sys.modules[fullname]


    def find_module(self, fullname, path=None):
        """This makes the class a finder. It is given the name of a module as well as
           the package that contains it (if applicable). It is expected to return a 
           loader for that module if it knows of one or None in which case other methods
           will be tried"""
        if fullname.split('.')[0] in self.packages:
            print "found {0}".format(fullname)
            return self
        else:
            return None


# This is a list of finder objects which is empty by defaule
# It is tried before anything else when a request to import a module is encountered.
sys.meta_path=[DynamicImporter('foo')]

from foo.bar import ThisShouldWork

Python包/模块懒洋洋地加载子模块

1 个答案:

更新