Question

阅读Executing Mach-O files的Apple文档后，它说：

OS X v10.1及更高版本的两级命名空间功能添加了模块名称作为其中定义的符号的符号名称的一部分它。 此方法可确保模块的符号名称不会与之冲突其他模块中使用的名称。

所以在我的例子中，我将 python2 和 python3 加载到同一个进程中。两个Python库（默认情况下）都是使用两级命名空间选项编译的。两个lib也通过dlopen（..）加载了RTLD_GLOBAL标志，因此具有相同名称的符号不应互相干扰，因为这两个模块具有不同的名称（python27和python36）。 / p>

示例：

#include <{...}/include/python2.7/Python.h>

int main(int argc, const char * argv[])
{
    auto* py3 = dlopen(".../python36", RTLD_GLOBAL | RTLD_NOW);
    if (py3 == nullptr)
        return 0;

    auto* py2 = dlopen(".../python27", RTLD_GLOBAL | RTLD_NOW);
    if (py2 == nullptr)
        return 0;

    auto* init = ((decltype(Py_Initialize)*)dlsym(py2, "Py_Initialize"));
    if (init)
    {
        init();
    }

    return 0;
}

问题是，在python2导入/path/to/python2/lib/lib-dynload/_locale.so之后，调用python3中的函数PyModule_GetDict。这是为什么？怎么会发生这种情况？两级命名空间不应该阻止吗？

P.S。 lib-dynload是一个在macOS上具有Python附加C模块的目录。我验证了python2环境中正确的_local.so lib被加载了。

修改

在做了一些实验之后，我看到第一个加载的python lib的符号总是获得更高的优先级，但不确定它是否适用于第一个加载的库或仍然是“未定义的行为”。

调用python27的Py_Initialize（） - 成功：

1. Loading python27 first
2. Loading python36 second
3. PYTHONHOME to python27
4. cal Py_Initialize() of python27

调用python27的Py_Initialize（） - 崩溃：

1. Loading python36 first
2. Loading python27 second
3. PYTHONHOME to python27
4. cal Py_Initialize() of python27

我反过来得到了相同的结果。

调用python36的Py_Initialize（） - 成功：

1. Loading python36 first
2. Loading python27 second
3. PYTHONHOME to python36
4. cal Py_Initialize() of python36

调用python36的Py_Initialize（） - 崩溃：

1. Loading python27 first
2. Loading python36 second
3. PYTHONHOME to python36
4. cal Py_Initialize() of python36

Answer 1

The symbols within libpython (e.g. libpython2.7.dylib) are being resolved correctly. For example, under the above described scenario, I see PyModule_GetDict() get called 155 times before the incorrectly resolved call.

The problem is that python itself is compiling shared libraries, and it's using dlopen() to load them. You can see the dlopen() happening by setting the environment variable PYTHONVERBOSE when running:

$ PYTHONVERBOSE=1 ./main 2>&1 | grep dlopen

which produces:

dlopen(".../lib/python2.7/lib-dynload/_locale.so", 2);

The 2 argument corresponds to RTL_NOW, but that doesn't matter too much. The issue is that this separate library isn't able to indicate that it's symbols should be resolved against the libpython2.7.dylib library. Yet, it does have several python symbols; in particular, this one that ends up causing the problem:

$ nm prefix/lib/python2.7/lib-dynload/_locale.so | grep GetDict
         U _PyModule_GetDict

So, when python dlopen()s the library, all it can do is resolve the symbol without the qualification. Apparently, the semantic of the dl functionality is to resolve such symbols based on the order the libraries are loaded, as you noted.

So, things work fine until we load _locale.so, as you can see from the following backtrace:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x50)
  * frame #0: 0x00000001003f3fc1 libpython3.6m.dylib`PyErr_FormatV [inlined] PyErr_Restore at errors.c:42 [opt]
    frame #1: 0x00000001003f3fb7 libpython3.6m.dylib`PyErr_FormatV [inlined] PyErr_Clear at errors.c:355 [opt]
    frame #2: 0x00000001003f3fb7 libpython3.6m.dylib`PyErr_FormatV(exception=0x00000001004cba18, format="%s:%d: bad argument to internal function", vargs=0x00007fff5fbfdcb0) at errors.c:841 [opt]
    frame #3: 0x00000001003f2c39 libpython3.6m.dylib`PyErr_Format(exception=<unavailable>, format=<unavailable>) at errors.c:860 [opt]
    frame #4: 0x0000000100358220 libpython3.6m.dylib`PyModule_GetDict(m=0x0000000101a5a868) at moduleobject.c:450 [opt]
    frame #5: 0x00000001000f491c _locale.so`init_locale at _localemodule.c:703 [opt]
    frame #6: 0x00000001018d1176 libpython2.7.dylib`_PyImport_LoadDynamicModule(name="_locale", pathname=".../lib/python2.7/lib-dynload/_locale.so", fp=<unavailable>) at importdl.c:53 [opt]

Also worth noting, _locale.so is just the first library to fail. If you got past it somehow, there are quite a few other libraries that potentially will have similar problems in .../lib/python2.7/lib-dynload.

MacOS

1 个答案: