我在使用boost.python公开的现有C ++库中集成boost.signals2时遇到问题。
我有一个用std::shared_ptr
暴露给python的类。
这个类应该能够在某些事件上提出一些信号。
因此,我暴露了connect_slot
函数,该函数以boost::python::object
为参数。如果我在连接一个插槽后直接发出一个信号,一切正常,但如果该类后来提升信号,我会收到分段错误。
我认为这可能与c ++ lib中的线程有关(它也使用了boost :: asio等。)
以下是一些代码段:
MyClass.h:
public:
typedef boost::signals2::signal<void (std::shared_ptr<int>)> signal_my_sig;
void connect_slot(boost::python::object const & slot);
private:
signal_my_sig m_sig;
MyClass.cpp:
void MyClass::connect_slot(boost::python::object const & slot) {
std::cout << "register shd" << std::endl;
m_sig.connect(slot);
m_sig(12345); // this works
}
void MyClass::some_later_event() {
m_sig(654321); // this does not work
}
我在python中使用这样的自定义python函数调用MyClass :: connect_slot函数:
def testfunc(some_int):
print("slot called")
m = myext.MyClass()
m.connect_slot(testfunc)
MyClass::some_later_event
中引发的分段错误的回溯(使用gdb)如下所示:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff3c37700 (LWP 20634)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff3c37700 (LWP 20634)]
0x00000000004f7480 in PyObject_Call ()
(gdb)
(gdb) backtrace
#0 0x00000000004f7480 in PyObject_Call ()
#1 0x00000000004f7aa6 in PyEval_CallObjectWithKeywords ()
#2 0x000000000049bd84 in PyEval_CallFunction ()
#3 0x00007ffff5375d9f in boost::python::call<boost::python::api::object, int>
(callable=0x7ffff7ed4578, a0=@0x7ffff3c35b34: 5)
at /usr/local/boost_1_55_0/boost/python/call.hpp:66
#4 0x00007ffff5374b81 in boost::python::api::object_operators<boost::python::api::object>::operator()<int> (this=0x9e3bf0, a0=@0x7ffff3c35b34: 5)
at /usr/local/boost_1_55_0/boost/python/object_call.hpp:19
#5 0x00007ffff5373658 in boost::detail::function::void_function_obj_invoker1<boost::python::api::object, void, int>::invoke (function_obj_ptr=..., a0=5)
at /usr/local/boost_1_55_0/boost/function/function_template.hpp:153
#6 0x00007ffff5378a3c in boost::function1<void, int>::operator() (
this=0x9e3be8, a0=5)
at /usr/local/boost_1_55_0/boost/function/function_template.hpp:767
#7 0x00007ffff53781f9 in boost::signals2::detail::call_with_tuple_args<boost::signals2::detail::void_type>::m_invoke<boost::function<void (int)>, 0u, int&>(void*, boost::function<void (int)>&, boost::signals2::detail::unsigned_meta_array<0u>, std::tuple<int&>) const (this=0x7ffff3c35c7f, func=..., args=...)
at /usr/local/boost_1_55_0/boost/signals2/detail/variadic_slot_invoker.hpp:92
有什么想法吗?
答案 0 :(得分:5)
如果从未明确管理Global Interpreter Lock(GIL)的C ++线程调用MyClass::some_later_event()
,则可能导致未定义的行为。
让我们考虑C ++线程与Python交互的情况。例如,可以将C ++线程设置为在MyClass
一段时间后调用MyClass.event_in(seconds, value)
的信号。
这个例子可以变得相当复杂,所以让我们从基础开始:Python的GIL。简而言之,GIL是解释器周围的互斥体。如果一个线程正在做任何影响python托管对象的引用计数的事情,那么它需要获得GIL。在GDB回溯中,Boost.Signals2库可能试图在没有GIL的情况下调用Python对象,从而导致崩溃。虽然管理GIL非常简单,但它可能会很快变得复杂。
首先,模块需要让Python初始化GIL以进行线程化。
BOOST_PYTHON_MODULE(example)
{
PyEval_InitThreads(); // Initialize GIL to support non-python threads.
// ...
}
为方便起见,我们创建一个简单的类来帮助管理GIL:
/// @brief RAII class used to lock and unlock the GIL.
class gil_lock
{
public:
gil_lock() { state_ = PyGILState_Ensure(); }
~gil_lock() { PyGILState_Release(state_); }
private:
PyGILState_STATE state_;
};
让我们确定C ++线程何时需要GIL:
boost::signals2::signal
可以制作连接对象的其他副本,就像调用concurrently信号时所做的那样。boost::signals2::signal
调用对象连接的Python。回调肯定会影响python对象。例如,提供给self
方法的__call__
参数将增加和减少对象的引用计数。MyClass
类。这是一个基于原始代码的基本模型类:
/// @brief Mockup class.
class MyClass
{
public:
/// @brief Connect a slot to the signal.
template <typename Slot>
void connect_slot(const Slot& slot)
{
signal_.connect(slot);
}
/// @brief Send an event to the signal.
void event(int value)
{
signal_(value);
}
private:
boost::signals2::signal<void(int)> signal_;
};
由于C ++线程可能正在调用MyClass
的信号,因此MyClass
的生命周期必须至少与线程一样长。实现这一目标的一个很好的选择是让Boost.Python使用MyClass
管理boost::shared_ptr
。
BOOST_PYTHON_MODULE(example)
{
PyEval_InitThreads(); // Initialize GIL to support non-python threads.
namespace python = boost::python;
python::class_<MyClass, boost::shared_ptr<MyClass>,
boost::noncopyable>("MyClass")
.def("event", &MyClass::event)
// ...
;
}
boost::signals2::signal
与python对象进行交互。 boost::signals2::signal
可以在调用时制作副本。另外,可能有C ++插槽连接到信号,因此在调用Python插槽时仅锁定GIL是理想的。但是,signal
不提供钩子以允许我们在创建插槽副本或调用插槽之前获取GIL。
为了避免让signal
创建boost::python::object
个插槽的副本,可以使用创建boost::python::object
副本的包装类,以便引用计数保持准确,并管理副本通过shared_ptr
。这允许signal
自由创建shared_ptr
的副本,而不是在没有GIL的情况下复制boost::python::object
。
此GIL安全插槽可以封装在辅助类中。
/// @brief Helepr type that will manage the GIL for a python slot.
///
/// @detail GIL management:
/// * Caller must own GIL when constructing py_slot, as
/// the python::object will be copy-constructed (increment
/// reference to the object)
/// * The newly constructed python::object will be managed
/// by a shared_ptr. Thus, it may be copied without owning
/// the GIL. However, a custom deleter will acquire the
/// GIL during deletion.
/// * When py_slot is invoked (operator()), it will acquire
/// the GIL then delegate to the managed python::object.
struct py_slot
{
public:
/// @brief Constructor that assumes the caller has the GIL locked.
py_slot(const boost::python::object& object)
: object_(
new boost::python::object(object), // GIL locked, so copy.
[](boost::python::object* object) // Delete needs GIL.
{
gil_lock lock;
delete object;
}
)
{}
// Use default copy-constructor and assignment-operator.
py_slot(const py_slot&) = default;
py_slot& operator=(const py_slot&) = default;
template <typename ...Args>
void operator()(Args... args)
{
// Lock the GIL as the python object is going to be invoked.
gil_lock lock;
(*object_)(args...);
}
private:
boost::shared_ptr<boost::python::object> object_;
};
辅助函数将暴露给Python以帮助调整类型。
/// @brief MyClass::connect_slot helper.
template <typename ...Args>
void MyClass_connect_slot(
MyClass& self,
boost::python::object object)
{
py_slot slot(object); // Adapt object to a py_slot for GIL management.
// Using a lambda here allows for the args to be expanded automatically.
// If bind was used, the placeholders would need to be explicitly added.
self.connect_slot([slot](Args... args) mutable { slot(args...); });
}
更新的绑定揭示了辅助函数:
python::class_<MyClass, boost::shared_ptr<MyClass>,
boost::noncopyable>("MyClass")
.def("connect_slot", &MyClass_connect_slot<int>)
.def("event", &MyClass::event)
// ...
;
线程的功能非常基本:它休眠然后调用信号。但是,了解GIL的背景非常重要。
/// @brief Sleep then invoke an event on MyClass.
template <typename ...Args>
void MyClass_event_in_thread(
boost::shared_ptr<MyClass> self,
unsigned int seconds,
Args... args)
{
// Sleep without the GIl.
std::this_thread::sleep_for(std::chrono::seconds(seconds));
// We do not want to hold the GIL while invoking or copying
// C++-specific slots connected to the signal. Thus, it is the
// responsibility of python slots to manage the GIL via the
// py_slot wrapper class.
self->event(args...);
}
/// @brief Function that will be exposed to python that will create
/// a thread to call the signal.
template <typename ...Args>
void MyClass_event_in(
boost::shared_ptr<MyClass> self,
unsigned int seconds,
Args... args)
{
// The caller may or may not have the GIL. Regardless, spawn off a
// thread that will sleep and then invoke an event on MyClass. The
// thread will not be joined so detach from it. Additionally, as
// shared_ptr is thread safe, copies of it can be made without the
// GIL.
std::thread(&MyClass_event_in_thread<Args...>, self, seconds, args...)
.detach();
}
请注意,MyClass_event_in_thread
可以表示为lambda,但在lambda中解压缩模板包在某些编译器上不起作用。
MyClass
绑定已更新。
python::class_<MyClass, boost::shared_ptr<MyClass>,
boost::noncopyable>("MyClass")
.def("connect_slot", &MyClass_connect_slot<int>)
.def("event", &MyClass::event)
.def("event_in", &MyClass_event_in<int>)
;
最终解决方案如下:
#include <thread> // std::thread, std::chrono
#include <boost/python.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/signals2/signal.hpp>
/// @brief Mockup class.
class MyClass
{
public:
/// @brief Connect a slot to the signal.
template <typename Slot>
void connect_slot(const Slot& slot)
{
signal_.connect(slot);
}
/// @brief Send an event to the signal.
void event(int value)
{
signal_(value);
}
private:
boost::signals2::signal<void(int)> signal_;
};
/// @brief RAII class used to lock and unlock the GIL.
class gil_lock
{
public:
gil_lock() { state_ = PyGILState_Ensure(); }
~gil_lock() { PyGILState_Release(state_); }
private:
PyGILState_STATE state_;
};
/// @brief Helepr type that will manage the GIL for a python slot.
///
/// @detail GIL management:
/// * Caller must own GIL when constructing py_slot, as
/// the python::object will be copy-constructed (increment
/// reference to the object)
/// * The newly constructed python::object will be managed
/// by a shared_ptr. Thus, it may be copied without owning
/// the GIL. However, a custom deleter will acquire the
/// GIL during deletion.
/// * When py_slot is invoked (operator()), it will acquire
/// the GIL then delegate to the managed python::object.
struct py_slot
{
public:
/// @brief Constructor that assumes the caller has the GIL locked.
py_slot(const boost::python::object& object)
: object_(
new boost::python::object(object), // GIL locked, so copy.
[](boost::python::object* object) // Delete needs GIL.
{
gil_lock lock;
delete object;
}
)
{}
// Use default copy-constructor and assignment-operator.
py_slot(const py_slot&) = default;
py_slot& operator=(const py_slot&) = default;
template <typename ...Args>
void operator()(Args... args)
{
// Lock the GIL as the python object is going to be invoked.
gil_lock lock;
(*object_)(args...);
}
private:
boost::shared_ptr<boost::python::object> object_;
};
/// @brief MyClass::connect_slot helper.
template <typename ...Args>
void MyClass_connect_slot(
MyClass& self,
boost::python::object object)
{
py_slot slot(object); // Adapt object to a py_slot for GIL management.
// Using a lambda here allows for the args to be expanded automatically.
// If bind was used, the placeholders would need to be explicitly added.
self.connect_slot([slot](Args... args) mutable { slot(args...); });
}
/// @brief Sleep then invoke an event on MyClass.
template <typename ...Args>
void MyClass_event_in_thread(
boost::shared_ptr<MyClass> self,
unsigned int seconds,
Args... args)
{
// Sleep without the GIL.
std::this_thread::sleep_for(std::chrono::seconds(seconds));
// We do not want to hold the GIL while invoking or copying
// C++-specific slots connected to the signal. Thus, it is the
// responsibility of python slots to manage the GIL via the
// py_slot wrapper class.
self->event(args...);
}
/// @brief Function that will be exposed to python that will create
/// a thread to call the signal.
template <typename ...Args>
void MyClass_event_in(
boost::shared_ptr<MyClass> self,
unsigned int seconds,
Args... args)
{
// The caller may or may not have the GIL. Regardless, spawn off a
// thread that will sleep and then invoke an event on MyClass. The
// thread will not be joined so detach from it. Additionally, as
// shared_ptr is thread safe, copies of it can be made without the
// GIL.
// Note: MyClass_event_in_thread could be expressed as a lambda,
// but unpacking a template pack within a lambda does not work
// on some compilers.
std::thread(&MyClass_event_in_thread<Args...>, self, seconds, args...)
.detach();
}
BOOST_PYTHON_MODULE(example)
{
PyEval_InitThreads(); // Initialize GIL to support non-python threads.
namespace python = boost::python;
python::class_<MyClass, boost::shared_ptr<MyClass>,
boost::noncopyable>("MyClass")
.def("connect_slot", &MyClass_connect_slot<int>)
.def("event", &MyClass::event)
.def("event_in", &MyClass_event_in<int>)
;
}
测试脚本:
from time import sleep
import example
def spam1(x):
print "spam1: ", x
def spam2(x):
print "spam2: ", x
c = example.MyClass()
c.connect_slot(spam1)
c.connect_slot(spam2)
c.event(123)
print "Sleeping"
c.event_in(3, 321)
sleep(5)
print "Done sleeping"
结果如下:
spam1: 123
spam2: 123
Sleeping
spam1: 321
spam2: 321
Done sleeping
答案 1 :(得分:0)
感谢Tanner Sansbury在this帖子上链接到他的答案。这解决了我的问题,除了我无法调用接受参数的信号。
我通过编辑py_slot类来解决这个问题:
struct py_slot {
public:
/// @brief Constructor that assumes the caller has the GIL locked.
py_slot(const boost::python::object& object)
: object_(new boost::python::object(object), // GIL locked, so copy.
py_deleter<boost::python::object>()) // Delete needs GIL.
{}
void operator()(SomeParamClass param) {
// Lock the gil as the python object is going to be invoked.
gil_lock lock;
(*object_)(param);
private:
boost::shared_ptr<boost::python::object> object_;
};
boost :: bind调用如下所示:
self->connect_client_ready(boost::bind(&py_slot<SomeParamClass>::operator(), py_slot<SomeParamClass>(object), _1)); // note the _1