腌制C扩展

时间:2017-08-28 19:21:08

标签: python c python-multiprocessing python-c-extension

与此问题有关 python: pickling c objects 我使用__reduce__函数扩展了我的Python C扩展:

static PyMethodDef Kraken_methods[] = {                                                                
    {"name", (PyCFunction)Kraken_name, METH_NOARGS, "Return the first and last name"},
    {"__reduce__", (PyCFunction)Kraken_reduce, METH_NOARGS, "Necessary for pickling objects"},
    {NULL}  /* Sentinel */                                                                              
}; 

我的__reduce__函数(Kraken_reduce)看起来像第一次尝试一样,如下所示:

static PyObject *Kraken_reduce(Kraken* self){               

char* x = "hello from reduce\n";
printf("reduce called !!!\n");
return x; 
}

当然,我会期待一种异常或由于返回错误类型而导致的异常。但是当我调用函数explicit

kr_mod = kr_module.kr_module(api_key, sec_key)
kr_mod.__reduce__()

我收到以下错误:

  File "pipe_multiprocessor.py", line 99, in <module>
    kr_mod.__reduce__()
  File "/home/stephan/anaconda3/lib/python3.6/copyreg.py", line 65, in _reduce_ex
    raise TypeError("can't pickle %s objects" % base.__name__)
TypeError: can't pickle kr_module objects

printf指令未执行。

另一方面,如果我在python中导入C扩展并在导入后定义__reduce__方法:

class kraken_mod(kr_module.kr_module):
    def __reduce__(self):
            return (self.__class__, (api_key, sec_key, ))

酸洗(出于多处理的目的)工作正常!

那么我做错了什么?我想在中定义__reduce__方法 C代码,以便我可以省略额外的in-python定义。

编辑:在我的工具链中发现错误导致该方法无法编译和执行后,我在__reduce__方法的定义中没有得到任何进一步的解释。

static PyObject *Kraken_reduce(kr_module* self){

    PyObject *getattr;
    PyObject *builtins;
    PyObject *tuple;

    _Py_IDENTIFIER(getattr);
    builtins = PyEval_GetBuiltins();

    getattr = _PyDict_GetItemId(builtins, &PyId_getattr);
    PyObject_Print(getattr, stdout, 0);

    printf("reduce called !!!\n");

    tuple = Py_BuildValue("O(ss)", getattr, "123", "456");
    return tuple;
}

当kr_module发送到另一个进程时,现在调用__reduce__方法。在新进程中抛出异常: (我在主线程中将异常拉回来)

Exception in target_0(): <class 'AttributeError'>
Exception in target_0(): 'str' object has no attribute '456'

任何人都可以帮助我吗?

1 个答案:

答案 0 :(得分:0)

我找到了解决问题的方法: 以下功能对我有用:

 static PyObject *Kraken_reduce(kr_module* self){

    PyObject *tuple;
    PyObject *obj;
    PyObject *attr;

    obj = (PyObject*)self;

    attr = PyObject_GetAttrString(obj, "__class__");

    tuple = Py_BuildValue("O(ss)", attr, self->kr_api->s_api_key, self->kr_api->s_sec_key);

    return tuple;
}

Github

提供了文件的完整描述