带有Tensorflow的Pyinstaller为_checkpoint_ops.so文件采用了错误的路径

时间:2017-09-14 13:19:03

标签: python tensorflow pyinstaller

我正在尝试使用TensorflowPyinstaller生成我的Python代码的可执行文件。可执行文件正确生成但是当我尝试运行它时,我收到以下错误:

Traceback (most recent call last):
  File "detection_init.py", line 14, in <module>
    import lib.tensorboxDetector as tensorboxDetector
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "lib/tensorboxDetector.py", line 26, in <module>
    from lib.train import build_forward
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "lib/train.py", line 4, in <module>
    import tensorflow.contrib.slim as slim
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "tensorflow/contrib/__init__.py", line 22, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "tensorflow/contrib/bayesflow/__init__.py", line 24, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "tensorflow/contrib/bayesflow/python/ops/csiszar_divergence.py", line 26, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "tensorflow/contrib/bayesflow/python/ops/csiszar_divergence_impl.py", line 42, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "tensorflow/contrib/framework/__init__.py", line 89, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "tensorflow/contrib/framework/python/ops/__init__.py", line 24, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "tensorflow/contrib/framework/python/ops/checkpoint_ops.py", line 32, in <module>
  File "tensorflow/contrib/util/loader.py", line 55, in load_op_library
  File "tensorflow/python/framework/load_library.py", line 64, in load_op_library
tensorflow.python.framework.errors_impl.NotFoundError: tensorflow/contrib/util/tensorflow/contrib/framework/python/ops/_checkpoint_ops.so: cannot open shared object file: No such file or directory
[11241] Failed to execute script detection_init

如果仔细观察,Pyinstaller期望目录_checkpoint_ops.so中的文件tensorflow/contrib/util/tensorflow/contrib/framework/python/ops/,但是没有这样的目录。 _checkpoint_ops.so位于tensorflow/contrib/framework/python/ops/。如何纠正这个错误?

1 个答案:

答案 0 :(得分:0)

将以下内容添加到您的规范文件中(找到tensorflow二进制文件并将它们添加到主二进制文件/文件目录中的.app中):

import os

tensorflow_location = '/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow'

tensorflow_binaries = []

for dir_name, sub_dir_list, fileList in os.walk(tensorflow_location): 
  for file in fileList:
    if file.endswith(".so"):
      full_file = dir_name + '/' + file
      print(full_file)
      tensorflow_binaries.append((full_file, '.'))

确保您还将二进制文件添加到Analysis

a = Analysis(...,
             binaries=tensorflow_binaries,
             ...)

并确保二进制文件包含在.exe / .app版本中。这是一个简单的示例:

pyz = PYZ(a.pure)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='TestApp',
          debug=True,
          strip=None,
          upx=True,
          console=True, icon='Test.icns')
app = BUNDLE(exe,
    a.scripts,
    a.binaries,
    a.zipfiles,
    a.datas,
    name='Test.app',
    icon='Test.icns',
    bundle_identifier='com.Test.TestApp',
    info_plist={
        'NSHighResolutionCapable': 'True'
            },
    )

您需要做的最后一件事是修改tensorflow load_library.py文件以指向.app运行时二进制文件的相对路径。

文件通常位于此处(相对于python安装):site-packages/tensorflow/python/framework/load_library.py

您需要添加resource_path函数并修改load_op_library函数以将file_name指向相对路径(并忽略通过子文件夹查找二进制文件)。

def resource_path(relative_path):
    """Due to pyinstaller changing root dir of project filenames need to be processed in order to open properly

    Parameters
    ----------
    relative_path : str
        String containing filename of project item (ex: models/bladder...h5)

    Returns
    -------
    path : str
        path relative to .exe on local computer
    """
    import sys
    import os

    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = sys._MEIPASS
    except Exception:
        base_path = os.path.abspath(".")

    path = os.path.join(base_path, relative_path)
    return path

def load_op_library(library_filename):
  """Loads a TensorFlow plugin, containing custom ops and kernels.

  Pass "library_filename" to a platform-specific mechanism for dynamically
  loading a library. The rules for determining the exact location of the
  library are platform-specific and are not documented here. When the
  library is loaded, ops and kernels registered in the library via the
  `REGISTER_*` macros are made available in the TensorFlow process. Note
  that ops with the same name as an existing op are rejected and not
  registered with the process.

  Args:
    library_filename: Path to the plugin.
      Relative or absolute filesystem path to a dynamic library file.

  Returns:
    A python module containing the Python wrappers for Ops defined in
    the plugin.

  Raises:
    RuntimeError: when unable to load the library or get the python wrappers.
  """ 
  # REMOVE AFTER PYINSTALLER USE
  library_filename = resource_path(library_filename.split('/')[-1])

这将使tensorflow通过pyinstaller应用程序的相对路径来获取二进制文件(请参阅:https://pythonhosted.org/PyInstaller/runtime-information.html#run-time-information)。

然后,Tensorflow将获取.spec文件打包的二进制文件。

请记住在Pyinstaller打包后取消注释library_filename = resource_path(library_filename.split('/')[-1])中的load_library.py行(否则Tensorflow在本地Python env中运行时将找不到二进制文件!)