在python中加载延迟模块

时间:2015-02-06 17:25:40

标签: python module lazy-loading

我试图组建一个系统来处理不明确存在的模块的延迟加载。基本上我有一个带有许多端点的http服务器,我不能提前知道,我想以编程方式提供导入。这些模块都有统一的方法签名,它们不会提前存在。

import lazy.route as test
import lazy.fake as test2

test('Does this exist?')  # This sends a post request.
test2("This doesn't exist.")  # Also sends a post request

我可以使用统一的装饰器处理这些导入所需的所有逻辑,我无法找到任何方式来装饰"在python中导入,或者实际上以任何编程方式与它们进行交互。

有没有人有这方面的经验?我一直在四处寻找,而我发现的最接近的是ast模块,在我目前的理解下,这会导致我目前真正糟糕的hacky实现(比如发现所有import语句并手动覆盖导入函数)

不是要寻找讲义,只是要开始查看的python代码库的一部分,或者是那些做过类似事情的人的例子。

1 个答案:

答案 0 :(得分:7)

我在google搜索中有点聪明,并设法找到专门解决这个问题的PEP,它恰好相对未知,可能是因为合理用途的子集非常狭窄。

我找到了一个很好的示例代码,展示了新的sys.meta_path实现。我已在下面发布了有关如何动态引导导入语句的信息。

import sys


class VirtualModule(object):

   def hello(self):
      return 'Hello World!'


class CustomImporter(object):

   virtual_name = 'my_virtual_module'

   def find_module(self, fullname, path):
      """This method is called by Python if this class
         is on sys.path. fullname is the fully-qualified
         name of the module to look for, and path is either
         __path__ (for submodules and subpackages) or None (for
         a top-level module/package).

         Note that this method will be called every time an import
         statement is detected (or __import__ is called), before
         Python's built-in package/module-finding code kicks in."""

      if fullname ==  self.virtual_name:

         # As per PEP #302 (which implemented the sys.meta_path protocol),
         # if fullname is the name of a module/package that we want to
         # report as found, then we need to return a loader object.
         # In this simple example, that will just be self.

         return self

      # If we don't provide the requested module, return None, as per
      # PEP #302.

      return None

   def load_module(self, fullname):
      """This method is called by Python if CustomImporter.find_module
         does not return None. fullname is the fully-qualified name
         of the module/package that was requested."""

      if fullname != self.virtual_name:
         # Raise ImportError as per PEP #302 if the requested module/package
         # couldn't be loaded. This should never be reached in this
         # simple example, but it's included here for completeness. :)
         raise ImportError(fullname)

      # PEP#302 says to return the module if the loader object (i.e,
      # this class) successfully loaded the module.
      # Note that a regular class works just fine as a module.
      return VirtualModule()


if __name__ == '__main__':

   # Add our import hook to sys.meta_path
   sys.meta_path.append(CustomImporter())

   # Let's use our import hook
   import my_virtual_module
   print my_virtual_module.hello()

完整的博文是here