scipy +(基于numpy.distutils)setup.py + ctypes:undefined symbol

时间:2018-05-02 22:44:15

标签: python c++ numpy scipy shared-libraries

简介

作为我之前问题here的后续内容,我仍然尝试:

  • 为其中一个scipy模块添加新功能(例如scipy.optimize
  • 基于C ++ - 代码
    • 定义了使用ctypes
    • 调用的接口函数
  • 通过setup.py
  • 中的scipy/optimize自动设置

代码合并

为了简单起见,让我们假设我们得到以下(愚蠢和)简单的C ++ - 代码:

extern "C" void hallo()
{
    int a = 0;
}

我们使用extern关键字来处理名称修改。

我们在scipy.optimize中引入新文件和文件夹:

scipy/optimize/__init__py
scipy/optimize/lbfgsb/...
scipy/optimize/lbfgsb.py
...
scipy/optimize/_mylib/README.md       # new
scipy/optimize/_mylib/LICENSE         # new
scipy/optimize/_mylib/src/debug.cpp   # new
scipy/optimize/mylib.py               # new

设置准备

我们在scipy/optimize/setup.py中准备模块设置:

from __future__ import division, print_function, absolute_import

from os.path import join

from scipy._build_utils import numpy_nodepr_api

def configuration(parent_package='',top_path=None):
    from numpy.distutils.misc_util import Configuration
    from numpy.distutils.system_info import get_info
    config = Configuration('optimize',parent_package, top_path)

    # MODIFICATION START
    # OTHER EXTENSIONS OMITTED 
    # INSPIRED BY scipy.spatial: ckdtree/src & setup.py

    mylib_src = ['debug.cpp']
    mylib_src = [join('_mylib', 'src', x) for x in mylib_src]

    mylib_headers = []
    mylib_headers = [join('_mylib', 'src', x) for x in mylib_headers]

    mylib_dep = mylib_headers + mylib_src

    config.add_extension('_mylib',
                         sources=mylib_src,
                         depends=mylib_dep,
                         include_dirs=[join('_mylib', 'src')])
    # MODIFICATION END    

    return config

if __name__ == '__main__':
    from numpy.distutils.core import setup
    setup(**configuration(top_path='').todict())

运行scipy install

现在安装scipy(来自base-dir):

python3 setup.py build_ext --inplace

有效,我们会在scipy/optimize/中看到这些共享库:

_lbfgsb.cpython-35m-x86_64-linux-gnu.so
...
_mylib.cpython-35m-x86_64-linux-gnu.so

让我们尝试使用它/看看scipy / optimize / mylib.py

正如在另一个question中看到的那样,我们可以获取lib(我们在scipy/optimize/mylib.py中)并获得我们想要使用的函数:

import scipy as scp
import numpy.ctypeslib as ctl
lib = ctl.load_library('_mylib', scp.optimize.__file__)
myfunc = lib.hallo

现在问题出现了:这失败了:

AttributeError: /.../_mylib.cpython-35m-x86_64-linux-gnu.so: undefined symbol: hallo

每手尝试:

import ctypes
lib = ctypes.CDLL('full path to above so')
myfun = lib.hallo

也失败了。

但是,检查(在我的Linux操作系统上):nm --defined-only _mylib.cpython-35m-x86_64-linux-gnu.so输出:

...
...
0000000000000530 t hallo
...

应该没问题。 (免责声明:我完全错过任何有关C ++链接的知识)。 编辑:也许不行。请参阅后面关于t vs. T的观察!

手动执行:此方法

转到scipy/optimize/_mylib/src

g++ -shared -fPIC debug.cpp -o mylib.so

后跟nm --defined-only mylib.so显示相同的功能,但有些t变为T

...
...
0000000000000600 T hallo
...

可能这就是原因,并且存在一些相关的一般性问题here

如上所述,这有效:

# in src
import numpy.ctypeslib as ctl
lib = ctl.load_library('mylib.so', '.')
lib.hallo
# < FuncPtr object at 0x....

但是在setup.py或我的来源需要做些什么才能使其发挥作用?

在运行scipy-install期间,输出如下所示:

building 'scipy.optimize._mylib' extension
compiling C++ sources
C compiler: x86_64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC

creating build/temp.linux-x86_64-3.5/scipy/optimize/_mylib
creating build/temp.linux-x86_64-3.5/scipy/optimize/_mylib/src
compile options: '-Iscipy/optimize/_mylib/src -I/usr/local/lib/python3.5/dist-packages/numpy-1.15.0.dev0+e4d678a-py3.5-linux-x86_64.egg/numpy/core/include -I/usr/include/python3.5m -c'
x86_64-linux-gnu-g++: scipy/optimize/_mylib/src/debug.cpp
scipy/optimize/_mylib/src/debug.cpp: In function ‘void hallo()’:
scipy/optimize/_mylib/src/debug.cpp:3:9: warning: unused variable ‘a’ [-Wunused-variable]
     int a = 0;
         ^
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.5/scipy/optimize/_mylib/src/debug.o -Lbuild/temp.linux-x86_64-3.5 -o scipy/optimize/_mylib.cpython-35m-x86_64-linux-gnu.so -Wl,--version-script=build/temp.linux-x86_64-3.5/link-version-scipy.optimize._mylib.map

0 个答案:

没有答案