无法加载基于CI的Dylib

时间:2016-06-10 16:18:34

标签: python rust cython travis-ci

我正在Travis CI上构建Rust二进制文件(liblonlat_bng.dylib),将其拉入Cython扩展(与Cython源.c / .pyx相同的目录),并对Travis进行测试CI(在不同的回购和构建中)。但是,Python包的测试失败了,我不确定原因:

----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/travis/build/urschrei/convertbng/venv/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/Users/travis/build/urschrei/convertbng/venv/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/Users/travis/build/urschrei/convertbng/venv/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/Users/travis/build/urschrei/convertbng/test/test_convertbng.py", line 15, in <module>
    from convertbng.cutil import convert_bng as cconvert_bng
ImportError: dlopen(/Users/travis/build/urschrei/convertbng/convertbng/cutil.so, 2): Library not loaded: /Users/travis/build/urschrei/lonlat_bng/target/x86_64-apple-darwin/release/liblonlat_bng.dylib
  Referenced from: /Users/travis/build/urschrei/convertbng/convertbng/cutil.so
  Reason: image not found

这是Travis的软件包的OSX构建输出:

Installing collected packages: convertbng
  Running setup.py develop for convertbng
    Running command /Users/travis/build/urschrei/convertbng/venv/bin/python2.7 -c "import setuptools, tokenize;__file__='/Users/travis/build/urschrei/convertbng/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps
    running develop
    running egg_info
    writing requirements to convertbng.egg-info/requires.txt
    writing convertbng.egg-info/PKG-INFO
    writing top-level names to convertbng.egg-info/top_level.txt
    writing dependency_links to convertbng.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'convertbng.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'convertbng.egg-info/SOURCES.txt'
    running build_ext
    building 'convertbng.cutil' extension
    creating build
    creating build/temp.macosx-10.11-x86_64-2.7
    creating build/temp.macosx-10.11-x86_64-2.7/convertbng
    clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -Iconvertbng -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c convertbng/cutil.c -o build/temp.macosx-10.11-x86_64-2.7/convertbng/cutil.o -O3
    [unused function warnings]
    creating build/lib.macosx-10.11-x86_64-2.7
    creating build/lib.macosx-10.11-x86_64-2.7/convertbng
    clang -bundle -undefined dynamic_lookup build/temp.macosx-10.11-x86_64-2.7/convertbng/cutil.o -L. -Lconvertbng -llonlat_bng -o build/lib.macosx-10.11-x86_64-2.7/convertbng/cutil.so
    copying build/lib.macosx-10.11-x86_64-2.7/convertbng/cutil.so -> convertbng
    Creating /Users/travis/build/urschrei/convertbng/venv/lib/python2.7/site-packages/convertbng.egg-link (link to .)
    Adding convertbng 0.4.14 to easy-install.pth file
    Installed /Users/travis/build/urschrei/convertbng
Successfully installed convertbng

这是Linux Travis输出,它成功地将dylib定位在相对路径上。请注意$ORIGIN的{​​{1}}参数,该参数不能在OSX上使用:

-R

其他细节:

  • 我的OS X和Travis都在使用XCode 7.3
  • 正在本地和Travis上使用相同的提交构建二进制文件
  • 如果我在本地构建二进制文件,则在我的包上运行Installing collected packages: convertbng Running setup.py develop for convertbng Running command /usr/bin/python -c "import setuptools, tokenize;__file__='/home/travis/build/urschrei/convertbng/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps running develop running egg_info writing requirements to convertbng.egg-info/requires.txt writing convertbng.egg-info/PKG-INFO writing top-level names to convertbng.egg-info/top_level.txt writing dependency_links to convertbng.egg-info/dependency_links.txt warning: manifest_maker: standard file '-c' not found reading manifest file 'convertbng.egg-info/SOURCES.txt' writing manifest file 'convertbng.egg-info/SOURCES.txt' running build_ext building 'convertbng.cutil' extension creating build creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/convertbng x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I. -Iconvertbng -I/usr/include/python2.7 -c convertbng/cutil.c -o build/temp.linux-x86_64-2.7/convertbng/cutil.o -O3 creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/convertbng x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/convertbng/cutil.o -L. -Lconvertbng -Wl,-R$ORIGIN -llonlat_bng -o build/lib.linux-x86_64-2.7/convertbng/cutil.so copying build/lib.linux-x86_64-2.7/convertbng/cutil.so -> convertbng Creating /usr/local/lib/python2.7/dist-packages/convertbng.egg-link (link to .) Adding convertbng 0.4.14 to easy-install.pth file Installed /home/travis/build/urschrei/convertbng Successfully installed convertbng ,测试通过
  • 我为Linux运行相同的安装,使用在Travis上使用相同提交构建的.so,并且测试通过
  • 如果我将Travis构建的.dylib复制到我的本地计算机,然后运行setup.py build_ext --inplace,则测试失败并出现与Travis相同的错误。

我对这是怎么回事感到非常困惑。我应该在二进制文件中看到什么吗? setup.py build_ext --inplace输出略有不同。例如:

本地:

otool -l

特拉维斯:

cmd LC_LOAD_DYLIB
cmdsize 56
name /usr/lib/libSystem.B.dylib (offset 24)
time stamp 2 Thu Jan  1 01:00:02 1970
current version 1226.10.1

链接到传递Linux作业:https://travis-ci.org/urschrei/convertbng/jobs/136730347
链接到失败的OSX作业:https://travis-ci.org/urschrei/convertbng/jobs/136730348

更新: 这绝对是因为库名。如果我使用cmd LC_LOAD_DYLIB cmdsize 56 name /usr/lib/libSystem.B.dylib (offset 24) time stamp 2 Thu Jan 1 00:00:02 1970 current version 1225.1.1 将dylib位置更改为install_name_tool

@loader_path/liblonlat_bng.dylib

Travis构建的可执行文件将在我的本地计算机上运行。但是,Travis似乎有一个损坏的install_name_tool -change /Users/travis/build/urschrei/lonlat_bng/target/x86_64-apple-darwin/release/liblonlat_bng.dylib @loader_path/liblonlat_bng.dylib convertbng/cutil.so 安装,我没有设法从install_name_tool将正确的调用传递给cutil.so。我已尝试将setup.py设置为生成

extra_link_args
['-Wl,-rpath,'+'@loader_path/liblonlat_bng.dylib']

但如果我这样做,我的测试仍然会失败。

更新2 :在使用货物构建时,我可以在链接阶段修复dylib的/usr/bin/clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/temp.macosx-10.6-intel-2.7/convertbng/cutil.o -L. -Lconvertbng -llonlat_bng -o build/lib.macosx-10.6-intel-2.7/convertbng/cutil.so -Wl,-rpath,@loader_path/liblonlat_bng.dylib

install_name

这很有效。但我怀疑我的RUSTFLAGS="-C link-args=-Wl,-install_name,@rpath/liblonlat_bng.dylib" cargo build --release setup.py设置错误:

Extension

2 个答案:

答案 0 :(得分:0)

使用install_name_tool有两种方法可以在不使用的情况下修复此 假设您的dylib与Cython扩展名在同一目录中(在本例中为cutil.so

  • 使用rpath支持编译Rust二进制文件,并在setup.py Extension实例中正确设置rpath:
    • 最简洁的方法是在rpath=true的{​​{1}}下设置[profile.release]See here
    • 您也可以调用cargo.toml
  • 灵活性较低的方法是编译Rust二进制文件并通过链接器标志传递路径。这会在路径中进行烘焙(尽管有动态参考),但是:
    • RUSTFLAGS="-C rpath" cargo build --release

在您的Rust dylib中启用cargo rustc --release -- -C link-args=-Wl,-install_name,@rpath/libname.dylib支持后,请修改rpath

setup.py个实例中设置(或添加)extra_link_args=["-Wl,-rpath", "-Wl,@loader_path/"]

如果您编译Cython扩展程序(例如使用Extension),然后在生成的setup.py build_ext --inplace上运行otool -l

.so

如果您希望Rust dylib在其他地方相对于您的Cython扩展程序,您需要相应调整 @rpath/liblonlat_bng.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1) 中的@rpath/@loader_path/路径。

注意:上述内容仅适用于OS X上的相对路径。在Linux上,您可以在setup.py extra_link_args=["-Wl,-rpath", "-Wl,$ORIGIN"]个实例中设置setup.py(如果您的Rust dylib在其他地方,相对于您的扩展程序,则相应地进行调整。)

答案 1 :(得分:0)

我遇到了同样的问题,我相信这是Distutils中的错误。我已提出拉动请求以解决此问题,并且还提出了一种解决方法。参见https://github.com/python/cpython/pull/12418

您可以在1. remove url and date time from top of the pdf 2. set left and right margins for the pdf page 3. set font size and font type for pdf page 中尝试以下内容吗?

setup.py