简介

Question

简介

我正在编写一个使用C语言编写的库的python应用程序。在C级别发生某些事件时，通常会调用Python回调。

这是我的python回调定义的一部分：

def callback(str1, str2, cdata, flag):
  print("PYTHON HANDLER")
  ...
  print(">>EXIT<<")

在将回调传递给C之前，将使用以下代码对其进行转换：

FuncType = ctypes.CFUNCTYPE(None, ctypes.c_char_p, ctypes.c_char_p,
                            ctypes.c_void_p, ctypes.c_bool)

ctype_function_wrapper = FuncType(callback)
cfunction_pointer = ctypes.cast(ctype_function_wrapper, ctypes.c_void_p)
py_callback_ptr = cfunction_pointer.value

然后，将 py_callback_ptr 传递到C库，并在其中保存为（void *）指针。

事件发生时，将调用C级 py_callback_ptr 。这是一段在C级执行此操作的代码：

...
printf("Acquire GIL\r\n");
PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
printf("Callback()\r\n");
((void (*)(char*, char*, DataStructure*, bool))py_callback_ptr)(
    str1, str2, value, bool_flag);
printf("Callback return\r\n");
PyGILState_Release(gstate);
printf("Release GIL\r\n");
...

在大多数情况下，它会按预期工作并调用回调。我在stdout中得到以下输出：

...
Acquire GIL
Callback()
PYTHON HANDLER
>>EXIT<<
Callback return
Release GIL
...

但是有时候我会将此输出与stderr输出混合在一起。它显示了回调周围的C代码正常执行，但回调本身未执行：

...
Acquire GIL
Callback()
SystemError: null argument to internal routine      # from stderr
Callback return
Release GIL
...

研究

我已经在Python和https://stackoverflow.com/a/52052063/736079

中寻找了SystemError

在解释器发现内部错误时引发，但情况看起来并不那么严重，以致于它放弃了所有希望。关联的值是一个字符串，用于指示出了什么问题（用低级术语表示）。

因此，这是在解释器代码中捕获的错误，该解释器代码将一些字符串写入stderr。我决定接听电话并进行调查。

调试

我试图在gdb中捕获此错误。我已经设置了一个断点，可以使用以下命令在stderr输出上停止程序执行：

(gdb) b write if 2==$rdi

还捕获了正常回调执行的回溯。他们在这里。

正常回溯的一部分：

#0  write () at ../sysdeps/unix/syscall-template.S:81
#1  0x0000000000437735 in _Py_write_impl (gil_held=1, count=4, buf=<optimized out>, fd=2) at Python/fileutils.c:1331
#2  _Py_write (fd=2, buf=0x961100, count=<optimized out>) at Python/fileutils.c:1393
#3  0x00000000005b73ce in _io_FileIO_write_impl (b=0x7ffff3916b00, b=0x7ffff3916b00, self=0x7ffff7f0c3a8) at ./Modules/_io/fileio.c:857
#4  _io_FileIO_write (self=0x7ffff7f0c3a8, arg=<optimized out>) at ./Modules/_io/clinic/fileio.c.h:251
#5  0x00000000004aa139 in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff4f9d828, args=args@entry=0x7ffff3916c40, nargs=nargs@entry=1, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:209
#6  0x0000000000451b68 in _PyObject_FastCallDict (func=0x7ffff4f9d828, args=0x7ffff3916c40, nargs=1, kwargs=0x0) at Objects/abstract.c:2313
#7  0x0000000000452b86 in PyObject_CallMethodObjArgs (callable=0x7ffff4f9d828, name=<optimized out>) at Objects/abstract.c:2759
#8  0x00000000005bb24a in _bufferedwriter_raw_write (self=self@entry=0x7ffff7fd0b48, start=<optimized out>, len=4) at ./Modules/_io/bufferedio.c:1853
#9  0x00000000005bbd0f in _bufferedwriter_flush_unlocked (self=self@entry=0x7ffff7fd0b48) at ./Modules/_io/bufferedio.c:1898
#10 buffered_flush_and_rewind_unlocked (self=self@entry=0x7ffff7fd0b48) at ./Modules/_io/bufferedio.c:827
#11 0x00000000005bbf27 in buffered_flush (self=0x7ffff7fd0b48, args=<optimized out>) at ./Modules/_io/bufferedio.c:854
#12 0x00000000004aa1ff in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff4f9d5a0, args=args@entry=0x7ffff3916f30, nargs=nargs@entry=0, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:192
#13 0x0000000000451b68 in _PyObject_FastCallDict (func=0x7ffff4f9d5a0, args=0x7ffff3916f30, nargs=0, kwargs=0x0) at Objects/abstract.c:2313
#14 0x0000000000452b86 in PyObject_CallMethodObjArgs (callable=0x7ffff4f9d5a0, name=<optimized out>) at Objects/abstract.c:2759
#15 0x00000000005c1924 in _io_TextIOWrapper_write_impl (text=0x7ffff7f30a40, self=0x7ffff7f7d708) at ./Modules/_io/textio.c:1367
#16 _io_TextIOWrapper_write (self=0x7ffff7f7d708, arg=<optimized out>) at ./Modules/_io/clinic/textio.c.h:216
#17 0x00000000004aa139 in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff4f9d5e8, args=args@entry=0x7ffff3917148, nargs=nargs@entry=1, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:209
#18 0x0000000000451b68 in _PyObject_FastCallDict (func=func@entry=0x7ffff4f9d5e8, args=args@entry=0x7ffff3917148, nargs=nargs@entry=1, kwargs=kwargs@entry=0x0) at Objects/abstract.c:2313
#19 0x000000000047a4cc in PyFile_WriteObject (flags=1, f=<optimized out>, v=0x7ffff7f30a40) at Objects/fileobject.c:150
#20 PyFile_WriteString (s=s@entry=0x5e4ac5 "\n", f=<optimized out>) at Objects/fileobject.c:174
#21 0x000000000053b103 in builtin_print (self=<optimized out>, args=0x7ffff65df748, kwds=<optimized out>) at Python/bltinmodule.c:1814
#22 0x00000000004aa3d9 in PyCFunction_Call (func=func@entry=0x7ffff7fcdf30, args=args@entry=0x7ffff65df748, kwds=kwds@entry=0x7ffff4f9d8b8) at Objects/methodobject.c:98
#23 0x0000000000548572 in do_call_core (kwdict=0x7ffff4f9d8b8, callargs=0x7ffff65df748, func=0x7ffff7fcdf30) at Python/ceval.c:5116
#24 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3404
#25 0x000000000053f401 in PyEval_EvalFrameEx (throwflag=0, f=0x7ffff4fa1048) at Python/ceval.c:754
#26 _PyEval_EvalCodeWithName (_co=0x7ffff7f089c0, globals=globals@entry=0x7ffff7f4c360, locals=locals@entry=0x0, args=<optimized out>, argcount=1, kwnames=0x0, kwargs=kwargs@entry=0x7ffff7ed4d60, kwcount=0, kwstep=kwstep@entry=1, 
    defs=0x0, defcount=defcount@entry=0, kwdefs=0x0, closure=0x0, name=name@entry=0x7ffff7f0f3b0, qualname=0x7ffff7f0f3b0) at Python/ceval.c:4166
#27 0x000000000053f6ff in fast_function (kwnames=0x0, nargs=<optimized out>, stack=<optimized out>, func=0x7ffff7f36e18) at Python/ceval.c:4992
#28 call_function (pp_stack=pp_stack@entry=0x7ffff3917550, oparg=<optimized out>, kwnames=kwnames@entry=0x0) at Python/ceval.c:4872
#29 0x000000000054138b in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335
#30 0x000000000053e791 in PyEval_EvalFrameEx (throwflag=0, f=0x7ffff7ed4bb8) at Python/ceval.c:754
#31 _PyFunction_FastCall (co=<optimized out>, args=<optimized out>, nargs=4, globals=globals@entry=0x7ffff7f4c360) at Python/ceval.c:4933
#32 0x000000000053f927 in fast_function (kwnames=0x0, nargs=<optimized out>, stack=<optimized out>, func=0x7ffff4f9ed08) at Python/ceval.c:4968
#33 call_function (pp_stack=pp_stack@entry=0x7ffff3917770, oparg=<optimized out>, kwnames=kwnames@entry=0x0) at Python/ceval.c:4872
#34 0x000000000054138b in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335
#35 0x000000000053f401 in PyEval_EvalFrameEx (throwflag=0, f=0x7ffff4fa0048) at Python/ceval.c:754
#36 _PyEval_EvalCodeWithName (_co=0x7ffff65d0810, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7ffff6569330, argcount=4, kwnames=kwnames@entry=0x0, kwargs=kwargs@entry=0x0, kwcount=kwcount@entry=0, 
    kwstep=kwstep@entry=2, defs=defs@entry=0x0, defcount=defcount@entry=0, kwdefs=kwdefs@entry=0x0, closure=closure@entry=0x7ffff7e9e208, name=name@entry=0x0, qualname=qualname@entry=0x0) at Python/ceval.c:4166
#37 0x00000000005401dd in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7ffff6569330, argcount=<optimized out>, kws=kws@entry=0x0, kwcount=kwcount@entry=0, 
    defs=defs@entry=0x0, defcount=defcount@entry=0, kwdefs=0x0, closure=0x7ffff7e9e208) at Python/ceval.c:4187
#38 0x000000000048101c in function_call (func=0x7ffff4f9eea0, arg=0x7ffff6569318, kw=0x0) at Objects/funcobject.c:604
#39 0x0000000000451820 in PyObject_Call (func=0x7ffff4f9eea0, args=args@entry=0x7ffff6569318, kwargs=<optimized out>) at Objects/abstract.c:2261
#40 0x0000000000540696 in PyEval_CallObjectWithKeywords (func=func@entry=0x7ffff4f9eea0, args=args@entry=0x7ffff6569318, kwargs=kwargs@entry=0x0) at Python/ceval.c:4771
#41 0x00000000004516e7 in PyObject_CallObject (o=o@entry=0x7ffff4f9eea0, a=a@entry=0x7ffff6569318) at Objects/abstract.c:2187
#42 0x00007ffff4d54d2c in _CallPythonObject (pArgs=<optimized out>, flags=257, converters=0x7ffff656a908, callable=0x7ffff4f9eea0, setfunc=0x0, restype=0x7ffff4f631d0 <ffi_type_void>, mem=0x7ffff3917c20)
    at /home/ns-sergeev/.pythons/src/Python-3.6.8/Modules/_ctypes/callbacks.c:234
#43 closure_fcn (cif=<optimized out>, resp=0x7ffff3917c20, args=0x7ffff3917a80, userdata=<optimized out>) at /home/ns-sergeev/.pythons/src/Python-3.6.8/Modules/_ctypes/callbacks.c:296
#44 0x00007ffff4b42a6b in ffi_closure_unix64_inner (closure=0x7ffff7ff2010, rvalue=0x7ffff3917c20, reg_args=0x7ffff3917b70, argp=0x7ffff3917c40 "P\r") at ../src/x86/ffi64.c:667
#45 0x00007ffff4b42de4 in ffi_closure_unix64 () at ../src/x86/unix64.S:229
...
C-library frames are omitted

回溯，发生回调调用错误：

#0  write () at ../sysdeps/unix/syscall-template.S:81
#1  0x0000000000437735 in _Py_write_impl (gil_held=1, count=47, buf=<optimized out>, fd=2) at Python/fileutils.c:1331
#2  _Py_write (fd=2, buf=0x961100, count=<optimized out>) at Python/fileutils.c:1393
#3  0x00000000005b73ce in _io_FileIO_write_impl (b=0x7ffff3916e80, b=0x7ffff3916e80, self=0x7ffff7f0c3a8) at ./Modules/_io/fileio.c:857
#4  _io_FileIO_write (self=0x7ffff7f0c3a8, arg=<optimized out>) at ./Modules/_io/clinic/fileio.c.h:251
#5  0x00000000004aa139 in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff4f9e7e0, args=args@entry=0x7ffff3916fc0, nargs=nargs@entry=1, 
    kwargs=kwargs@entry=0x0) at Objects/methodobject.c:209
#6  0x0000000000451b68 in _PyObject_FastCallDict (func=0x7ffff4f9e7e0, args=0x7ffff3916fc0, nargs=1, kwargs=0x0) at Objects/abstract.c:2313
#7  0x0000000000452b86 in PyObject_CallMethodObjArgs (callable=0x7ffff4f9e7e0, name=<optimized out>) at Objects/abstract.c:2759
#8  0x00000000005bb24a in _bufferedwriter_raw_write (self=self@entry=0x7ffff7fd0b48, start=<optimized out>, len=47) at ./Modules/_io/bufferedio.c:1853
#9  0x00000000005bbd0f in _bufferedwriter_flush_unlocked (self=self@entry=0x7ffff7fd0b48) at ./Modules/_io/bufferedio.c:1898
#10 buffered_flush_and_rewind_unlocked (self=self@entry=0x7ffff7fd0b48) at ./Modules/_io/bufferedio.c:827
#11 0x00000000005bbf27 in buffered_flush (self=0x7ffff7fd0b48, args=<optimized out>) at ./Modules/_io/bufferedio.c:854
#12 0x00000000004aa1ff in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff4f9e558, args=args@entry=0x7ffff39172b0, nargs=nargs@entry=0, 
    kwargs=kwargs@entry=0x0) at Objects/methodobject.c:192
#13 0x0000000000451b68 in _PyObject_FastCallDict (func=0x7ffff4f9e558, args=0x7ffff39172b0, nargs=0, kwargs=0x0) at Objects/abstract.c:2313
#14 0x0000000000452b86 in PyObject_CallMethodObjArgs (callable=0x7ffff4f9e558, name=<optimized out>) at Objects/abstract.c:2759
#15 0x00000000005c1924 in _io_TextIOWrapper_write_impl (text=0x7ffff7f30a40, self=0x7ffff7f7d708) at ./Modules/_io/textio.c:1367
#16 _io_TextIOWrapper_write (self=0x7ffff7f7d708, arg=<optimized out>) at ./Modules/_io/clinic/textio.c.h:216
#17 0x00000000004aa139 in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff4f9e5a0, args=args@entry=0x7ffff39174c8, nargs=nargs@entry=1, 
    kwargs=kwargs@entry=0x0) at Objects/methodobject.c:209
#18 0x0000000000451b68 in _PyObject_FastCallDict (func=func@entry=0x7ffff4f9e5a0, args=args@entry=0x7ffff39174c8, nargs=nargs@entry=1, 
    kwargs=kwargs@entry=0x0) at Objects/abstract.c:2313
#19 0x000000000047a4cc in PyFile_WriteObject (flags=1, f=0x7ffff7f30a40, v=0x7ffff7f30a40) at Objects/fileobject.c:150
#20 PyFile_WriteString (s=s@entry=0x5e4ac5 "\n", f=f@entry=0x7ffff7f7d708) at Objects/fileobject.c:174
#21 0x0000000000425790 in print_exception (value=<optimized out>, f=0x7ffff7f7d708) at Python/pythonrun.c:819
#22 print_exception_recursive (f=f@entry=0x7ffff7f7d708, value=value@entry=0x7ffff4f9a1a8, seen=seen@entry=0x7ffff650ee48) at Python/pythonrun.c:891
---Type <return> to continue, or q <return> to quit---
#23 0x0000000000426683 in PyErr_Display (exception=<optimized out>, value=0x7ffff4f9a1a8, tb=<optimized out>) at Python/pythonrun.c:925
#24 0x000000000043199a in sys_excepthook (self=<optimized out>, args=<optimized out>) at ./Python/sysmodule.c:230
#25 0x00000000004aa1ac in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff7f61f78, args=args@entry=0x7ffff39176c0, nargs=nargs@entry=3, 
    kwargs=kwargs@entry=0x0) at Objects/methodobject.c:234
#26 0x0000000000451b68 in _PyObject_FastCallDict (func=0x7ffff7f61f78, args=args@entry=0x7ffff39176c0, nargs=nargs@entry=3, kwargs=kwargs@entry=0x0)
    at Objects/abstract.c:2313
#27 0x00000000004267f5 in PyErr_PrintEx (set_sys_last_vars=set_sys_last_vars@entry=1) at Python/pythonrun.c:669
#28 0x0000000000426a9a in PyErr_Print () at Python/pythonrun.c:532
#29 0x00007ffff4d54b78 in PrintError (msg=msg@entry=0x7ffff4d5bc7a "BUG: PySequence_Length")
    at /home/ns-sergeev/.pythons/src/Python-3.6.8/Modules/_ctypes/callbacks.c:91
#30 0x00007ffff4d54e8e in _CallPythonObject (pArgs=0x7ffff3917a80, flags=4353, converters=0x0, callable=0x0, setfunc=0x0, 
    restype=0x7ffff4f631d0 <ffi_type_void>, mem=0x7ffff3917c20) at /home/ns-sergeev/.pythons/src/Python-3.6.8/Modules/_ctypes/callbacks.c:149
#31 closure_fcn (cif=<optimized out>, resp=0x7ffff3917c20, args=0x7ffff3917a80, userdata=<optimized out>)
    at /home/ns-sergeev/.pythons/src/Python-3.6.8/Modules/_ctypes/callbacks.c:296
#32 0x00007ffff4b42a6b in ffi_closure_unix64_inner (closure=0x7ffff7ff0cc0, rvalue=0x7ffff3917c20, reg_args=0x7ffff3917b70, 
    argp=0x7ffff3917c40 "\240\021\243") at ../src/x86/ffi64.c:667
#33 0x00007ffff4b42de4 in ffi_closure_unix64 () at ../src/x86/unix64.S:229
...
C-library frames are omitted

初步发现

_CallPythonObject的参数转换器和可调用似乎不正确（由于某些原因已更改或损坏）。

第42帧来自正常回溯。普通值和调用回调。

#42 0x00007ffff4d54d2c in _CallPythonObject (pArgs=<optimized out>, flags=257,
    converters=0x7ffff656a908, callable=0x7ffff4f9eea0, setfunc=0x0,
    restype=0x7ffff4f631d0 <ffi_type_void>, mem=0x7ffff3917c20)

来自错误回溯的第30帧。零值和放弃的回调。

#30 0x00007ffff4d54e8e in _CallPythonObject (pArgs=0x7ffff3917a80, flags=4353, 
    converters=0x0, callable=0x0, setfunc=0x0,
    restype=0x7ffff4f631d0 <ffi_type_void>, mem=0x7ffff3917c20)

有人可以帮助解决此问题或建议进一步挖掘和调试它的方法吗？

Answer 1

您应该将cfunction_pointer而不是py_callback_ptr = cfunction_pointer.value传递到您的媒体库。

最小工作示例：

cside.h：

#pragma once

#ifdef __cplusplus
extern "C" {
#endif

typedef struct DataStructure {
  int a;
} DataStructure;

typedef void (*callback)(char*, char*, DataStructure*, bool);

typedef void (*pcallback)(char*, char*, void*, bool);

int calc0(int, callback cb);

int calc1(int, pcallback cb);


#ifdef __cplusplus
}
#endif

请注意，在C代码中的任何位置，DataStructure被void替换时，都不需要定义签名。无论如何，我都将其包括在内，以便在您的问题中使用类似的投射。

cside.cpp：

#include "cside.h"
#include "Python.h"

const char cstring1[] = "foo";
const char cstring2[] = "bar";

const bool bool_flag = false;

char* str1 = const_cast<char*>(&cstring1[0]);
char* str2 = const_cast<char*>(&cstring2[0]);

int calc0(int k, callback cb) {
  printf("Acquire GIL\r\n");
  PyGILState_STATE gstate;
  gstate = PyGILState_Ensure();
  printf("Callback()\r\n");

  DataStructure s;

  // Execute callback with the signature known in C
  cb(str1, str2, &s, bool_flag);

  printf("Callback return\r\n");
  PyGILState_Release(gstate);
  printf("Release GIL\r\n");
  return k+2;
}

int calc1(int k, pcallback cb) {
  printf("Acquire GIL\r\n");
  PyGILState_STATE gstate;
  gstate = PyGILState_Ensure();
  printf("Callback()\r\n");

  DataStructure s;

  // Cast to correct signature
  ((void (*)(char*, char*, DataStructure*, bool))cb)(
      str1, str2, &s, bool_flag);

  printf("Callback return\r\n");
  PyGILState_Release(gstate);
  printf("Release GIL\r\n");
  return k+2;
}

要编译：

g++ -I. -fPIC -shared -o libcside.so cside.cpp -I/usr/include/python2.7 -lpython2.7

在Python中进行测试：

import ctypes

libc = ctypes.CDLL('libcside.so')

def callback(str1, str2, cdata, flag):
  print("PYTHON HANDLER")
  print(">>EXIT<<")

FuncType = ctypes.CFUNCTYPE(None, ctypes.c_char_p, ctypes.c_char_p,
                            ctypes.c_void_p, ctypes.c_bool)

ctype_function_wrapper = FuncType(callback)
cfunction_pointer = ctypes.cast(ctype_function_wrapper, ctypes.c_void_p)
py_callback_ptr = cfunction_pointer.value

libc.calc0(2, cfunction_pointer)
libc.calc1(2, cfunction_pointer)

#libc.calc(2, py_callback_ptr) # segfaults

从C调用python回调时，如何解决“ SystemError：内部例程的空参数”错误

简介

研究

调试

初步发现

1 个答案: