卸载依赖项的示例

Question

我正在从我的python脚本中调用一个so文件。据我了解，我真的不需要释放使用ctypes在python中打开的共享库。但是，在我的so文件代码中，它dlopen另一个so文件并且不执行dlclose（）。在这种情况下，从python端使用安全吗？我不必释放在ctypes内部打开的共享库loade so file吗？

Answer 1

自己清洁 规则始终适用（尽管现代技术会为您提供清洁方面的帮助）。

[Python 3.5]: ctypes - A foreign function library for Python包含许多有用的信息，应该成为您的朋友。

ctypes 使用 dlopen 加载 .dll 。正如我所注意到的，它不调用相应的 dlclose ，这意味着 .dll （以及已加载的所有从属文件）加载时）将保留在内存中，直到该进程终止（或直到明确卸载）为止。

来自[man7]: DLOPEN(3)：

如果由文件名指定的对象具有对其他共享对象的依赖关系，则动态链接程序也会使用相同的规则自动加载这些依赖关系。（如果这些对象又具有依赖关系，则依次进行此过程，依此类推。）
...
如果使用 dlopen（）再次加载相同的共享对象，则返回相同的对象句柄。动态链接器维护对象句柄的引用计数，因此直到对其调用 dlclose（）的次数达到 dlopen（）的次数之后，才会释放动态加载的共享对象。成功了。任何初始化返回（请参见下文）仅被调用一次。

因此，我认为您不会遇到问题（当然，一切都取决于上下文）。如您所见，多次加载一个库实际上并不会每次都加载，因此用尽内存的机会非常小（除非您正在加载大量不同的 .dll s ，每个都有很多不同的依赖项。

我能想到的一种情况是加载使用另一个 .dll 中的符号的 .dll 。如果该符号也在之前加载的另一个（ 3 ^rd ） .dll 中定义，则代码的行为将与预期的不同。

无论如何，您可以手动卸载（或更好：减少其 refcount ）一个 .dll （我不确定这是否适合推荐的方式或最佳实践），如下面的示例所示。

dll.c ：

#include <stdio.h>


int test() {
    printf("[%s] (%d) - [%s]\n", __FILE__, __LINE__, __FUNCTION__);
    return 0;
}

code.py ：

import sys
from ctypes import CDLL, \
    c_int, c_void_p


DLL = "./dll.so"

dlclose_func = CDLL(None).dlclose  # This WON'T work on Win
dlclose_func.argtypes = [c_void_p]


def _load_dll(dll_name):
    dll_dll = CDLL(dll_name)
    print("{:}".format(dll_dll))
    return dll_dll


def _load_test_func(dll):
    test_func = dll.test
    test_func.restype = c_int
    return test_func


def main():
    print("Loading a dll via `ctypes`, then delete the object. The dll is not unloaded. Call `dlclose` to unload. A 2nd call will fail.")
    dll_dll = _load_dll(DLL)
    dll_handle = dll_dll._handle
    del dll_dll
    print("{:} returned {:d}".format(dlclose_func.__name__, dlclose_func(dll_handle)))  # Even if the ctypes dll object was destroyed, the dll wasn't unloaded
    print("{:} returned {:d}".format(dlclose_func.__name__, dlclose_func(dll_handle)))  # A new dlclose call will fail

    print("\nUse `ctypes` to load the dll twice. The dll is not actually loaded only the 1st time (both have the same handle), but its ref count is increased. `dlclose` must be also called twice.")
    dll0_dll = _load_dll(DLL)
    dll1_dll = _load_dll(DLL)
    print("{:} returned {:d}".format(dlclose_func.__name__, dlclose_func(dll0_dll._handle)))
    print("{:} returned {:d}".format(dlclose_func.__name__, dlclose_func(dll1_dll._handle)))
    print("{:} returned {:d}".format(dlclose_func.__name__, dlclose_func(dll1_dll._handle)))

    print("\nLoad a dll via `ctypes`, and load one of its funcs. Try calling it before and after unloading the dll.")
    dll_dll = _load_dll(DLL)
    test_func = _load_test_func(dll_dll)
    print("{:} returned {:d}".format(test_func.__name__, test_func()))
    print("{:} returned {:d}".format(dlclose_func.__name__, dlclose_func(dll_dll._handle)))
    print("{:} returned {:d}".format(test_func.__name__, test_func()))  # Comment this line as it would segfault !!!



if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

输出：

[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q052179325]> ls
code.py  dll.c
[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q052179325]> gcc -fPIC -shared -o dll.so dll.c
[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q052179325]> python3 ./code.py
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux

Loading a dll via `ctypes`, then delete the object. The dll is not unloaded. Call `dlclose` to unload. A 2nd call will fail.
<CDLL './dll.so', handle 1d7aa20 at 0x7fa38715f240>
dlclose returned 0
dlclose returned -1

Use `ctypes` to load the dll twice. The dll is not actually loaded only the 1st time (both have the same handle), but its ref count is increased. `dlclose` must be also called twice.
<CDLL './dll.so', handle 1de2c80 at 0x7fa38715f240>
<CDLL './dll.so', handle 1de2c80 at 0x7fa38715f278>
dlclose returned 0
dlclose returned 0
dlclose returned -1

Load a dll via `ctypes`, and load one of its funcs. Try calling it before and after unloading the dll.
<CDLL './dll.so', handle 1de39c0 at 0x7fa38715f8d0>
[dll.c] (5) - [test]
test returned 0
dlclose returned 0
Segmentation fault (core dumped)

Answer 2

卸载依赖项的示例

已在Linux Fedora 32，Python 3.7.6（anaconda），ctypes 1.1.0，g ++ 10.2.1中进行了测试。依赖性为OpenCv（4.2版）。此处有更多详细信息：How can I unload a DLL using ctypes in Python?

code.cpp

#include <opencv2/core/core.hpp>
#include <iostream> 


extern "C" int my_fct(int n)
{
    cv::Mat1b mat = cv::Mat1b(10,8,(unsigned char) 1 );  // change 1 to test unloading
    
    return mat(0,1) * n;
}

编译为 g++ code.cpp -shared -fPIC -Wall -std=c++17 -I/usr/include/opencv4 -lopencv_core -o so_opencv.so

Python代码

from sys import platform
import ctypes


class CtypesLib:

    def __init__(self, fp_lib, dependencies=[]):
        self._dependencies = [CtypesLib(fp_dep) for fp_dep in dependencies]

        if platform == "linux" or platform == "linux2":  # Linux
            self._dlclose_func = ctypes.cdll.LoadLibrary('').dlclose
            self._dlclose_func.argtypes = [ctypes.c_void_p]
            self._ctypes_lib = ctypes.cdll.LoadLibrary(fp_lib)
        elif platform == "win32":  # Windows
            self._ctypes_lib = ctypes.WinDLL(fp_lib)

        self._handle = self._ctypes_lib._handle

    def __getattr__(self, attr):
        return self._ctypes_lib.__getattr__(attr)

    def __del__(self):
        for dep in self._dependencies:
            del dep

        del self._ctypes_lib

        if platform == "linux" or platform == "linux2":  # Linux
            self._dlclose_func(self._handle)
        elif platform == "win32":  # Windows
            ctypes.windll.kernel32.FreeLibrary(self._handle)


fp_lib = './so_opencv.so'

ctypes_lib = CtypesLib(fp_lib, ['/usr/lib64/libopencv_core.so'])

valIn = 1
ctypes_lib.my_fct.argtypes = [ctypes.c_int]
ctypes_lib.my_fct.restype = ctypes.c_int
valOut = ctypes_lib.my_fct(valIn)
print(valIn, valOut)

del ctypes_lib

卸载共享库中的ctypes内的共享库

2 个答案:

卸载依赖项的示例