在Boost Python中包装数组

时间:2013-09-18 20:48:24

标签: c++ python boost boost-python

我有一系列C ++结构,我试图用boost python包装。当这些结构包含数组时,我遇到了困难。我试图以最小的开销执行此操作,不幸的是我无法对结构本身进行任何修改。所以比如说我有

struct Foo
{
    int vals[3];
};

我希望能够在python中访问它,如下所示:

f = Foo()
f.vals[0] = 10
print f.vals[0]

现在我正在使用一系列get / set函数,但这些函数非常不优雅且与访问其他非数组成员不一致。这是我目前的解决方案:

int getVals (Foo f, int index) { return f.vals[index]; }
void setVals (Foo& f, int index, int value) { f.vals[index] = value; }

boost::python::class_< Foo > ( "Foo", init<>() )
    .def ( "getVals", &getVals )
    .def ( "setVals", &setVals );

我没有get / set函数(因为在某些情况下我需要实现自定义get或set操作)但是我不确定如何合并[]运算符来访问数组的元素。在其他可以使用[]运算符访问的类中,我可以使用 _ getitem _ _ setitem _ < / strong>哪些都运作得很好,但我不确定如果有可能,我会如何与班级成员一起做这件事。

3 个答案:

答案 0 :(得分:19)

对于这样一个相对简单的问题,答案变得相当复杂。在提供解决方案之前,首先要检查问题的深度:

 f = Foo()
 f.vals[0] = 10

f.vals返回一个提供__getitem____setitem__方法的中间对象。为了支持Boost.Python,需要为每种类型的数组公开辅助类型,这些类型将提供索引支持。

语言之间的一个细微差别是对象生命周期。请考虑以下事项:

 f = Foo()
 v = f.vals
 f = None
 v[0] = 10

Python对象的生命周期通过引用计数进行管理,f不拥有vals引用的对象。因此,即使在f设置为f时销毁None引用的对象,v引用的对象仍然存活。这与需要公开的C ++ Foo类型形成对比,因为Foo拥有vals引用的内存。使用Boost.Python,f.vals返回的辅助对象需要延长f引用的对象的生命周期。


检查问题后,让我们开始解决方案。以下是需要公开的基本数组:

struct Foo
{
  int vals[3];
  boost::array<std::string, 5> strs;

  Foo()  { std::cout << "Foo()"  << std::endl; }
  ~Foo() { std::cout << "~Foo()" << std::endl; }
};

int more_vals[2];

Foo::valsFoo::strs的辅助类型需要提供最小的开销,同时支持索引。这是在array_proxy

中完成的
/// @brief Type that proxies to an array.
template <typename T>
class array_proxy
{
public:
  // Types
  typedef T           value_type;
  typedef T*          iterator;
  typedef T&          reference;
  typedef std::size_t size_type;

  /// @brief Empty constructor.
  array_proxy()
    : ptr_(0),
      length_(0)
  {}

  /// @brief Construct with iterators.
  template <typename Iterator>
  array_proxy(Iterator begin, Iterator end)
    : ptr_(&*begin),
      length_(std::distance(begin, end))
  {}

  /// @brief Construct with with start and size.
  array_proxy(reference begin, std::size_t length)
    : ptr_(&begin),
      length_(length)
  {}

  // Iterator support.
  iterator begin()               { return ptr_; }
  iterator end()                 { return ptr_ + length_; }

  // Element access.
  reference operator[](size_t i) { return ptr_[i]; }

  // Capacity.
  size_type size()               { return length_; }

private:
  T* ptr_;
  std::size_t length_;
};

完成辅助类型后,剩下的部分是添加在Python中向索引功能公开辅助类型的功能。 Boost.Python的indexing_suite提供了通过基于策略的方法为暴露类型添加索引支持的钩子。下面的ref_index_suite课程是符合DerivedPolicies类型要求的政策类:

/// @brief Policy type for referenced indexing, meeting the DerivedPolicies
///        requirement of boost::python::index_suite.
/// 
/// @note Requires Container to support:
///          - value_type and size_type types,
///          - value_type is default constructable and copyable,
///          - element access via operator[],
///          - Default constructable, iterator constructable,
///          - begin(), end(), and size() member functions
template <typename Container>
class ref_index_suite
  : public boost::python::indexing_suite<Container,
      ref_index_suite<Container> >

{
public:

  typedef typename Container::value_type data_type;
  typedef typename Container::size_type  index_type;
  typedef typename Container::size_type  size_type;

  // Element access and manipulation.

  /// @brief Get element from container.
  static data_type&
  get_item(Container& container, index_type index)
  {
    return container[index];
  }

  /// @brief Set element from container.
  static void
  set_item(Container& container, index_type index, const data_type& value)
  {
    container[index] = value;
  }

  /// @brief Reset index to default value.
  static void
  delete_item(Container& container, index_type index)
  {
    set_item(container, index, data_type());
  };

  // Slice support.

  /// @brief Get slice from container.
  ///
  /// @return Python object containing
  static boost::python::object
  get_slice(Container& container, index_type from, index_type to)
  {
    using boost::python::list;
    if (from > to) return list();

    // Return copy, as container only references its elements.
    list list;
    while (from != to) list.append(container[from++]);
    return list;
  };

  /// @brief Set a slice in container with a given value.
  static void
  set_slice(
    Container& container, index_type from,
    index_type to, const data_type& value
  )
  {
    // If range is invalid, return early.
    if (from > to) return;

    // Populate range with value.
    while (from < to) container[from++] = value;
  }

  /// @brief Set a slice in container with another range.
  template <class Iterator>
  static void
  set_slice(
    Container& container, index_type from,
    index_type to, Iterator first, Iterator last
  )
  {
    // If range is invalid, return early.
    if (from > to) return;

    // Populate range with other range.
    while (from < to) container[from++] = *first++;   
  }

  /// @brief Reset slice to default values.
  static void
  delete_slice(Container& container, index_type from, index_type to)
  {
    set_slice(container, from, to, data_type());
  }

  // Capacity.

  /// @brief Get size of container.
  static std::size_t
  size(Container& container) { return container.size(); }

  /// @brief Check if a value is within the container.
  template <class T>
  static bool
  contains(Container& container, const T& value)
  {
    return std::find(container.begin(), container.end(), value)
        != container.end();
  }

  /// @brief Minimum index supported for container.
  static index_type
  get_min_index(Container& /*container*/)
  {
      return 0;
  }

  /// @brief Maximum index supported for container.
  static index_type
  get_max_index(Container& container)
  {
    return size(container);
  }

  // Misc.

  /// @brief Convert python index (could be negative) to a valid container
  ///        index with proper boundary checks.
  static index_type
  convert_index(Container& container, PyObject* object)
  {
    namespace python = boost::python;
    python::extract<long> py_index(object);

    // If py_index cannot extract a long, then type the type is wrong so
    // set error and return early.
    if (!py_index.check()) 
    {
      PyErr_SetString(PyExc_TypeError, "Invalid index type");
      python::throw_error_already_set(); 
      return index_type();
    }

    // Extract index.
    long index = py_index();

    // Adjust negative index.
    if (index < 0)
        index += container.size();

    // Boundary check.
    if (index >= long(container.size()) || index < 0)
    {
      PyErr_SetString(PyExc_IndexError, "Index out of range");
      python::throw_error_already_set();
    }

    return index;
  }
};

每个辅助类型都需要通过Boost.Python与boost::python::class_<...>公开。这可能有点单调乏味,因此单个辅助功能将有条件地注册类型。

/// @brief Conditionally register a type with Boost.Python.
template <typename T>
void register_array_proxy()
{
  typedef array_proxy<T> proxy_type;

  // If type is already registered, then return early.
  namespace python = boost::python;
  bool is_registered = (0 != python::converter::registry::query(
    python::type_id<proxy_type>())->to_python_target_type());
  if (is_registered) return;

  // Otherwise, register the type as an internal type.
  std::string type_name = std::string("_") + typeid(T).name();
  python::class_<proxy_type>(type_name.c_str(), python::no_init)
    .def(ref_index_suite<proxy_type>());
}

此外,模板参数推导将用于为用户提供简单的API:

/// @brief Create a callable Boost.Python object from an array.
template <typename Array>
boost::python::object make_array(Array array)
{
  // Deduce the array_proxy type by removing all the extents from the
  // array.
  ...

  // Register an array proxy.
  register_array_proxy<...>();
}

从Python访问时,Foo::vals需要从int[3]转换为array_proxy<int>。模板类可以用作创建适当类型的array_proxy的仿函数。下面的array_proxy_getter提供了此功能。

/// @brief Functor used used convert an array to an array_proxy for
///        non-member objects.
template <typename NativeType,
          typename ProxyType>
struct array_proxy_getter
{
public:
  typedef NativeType native_type;
  typedef ProxyType  proxy_type;

  /// @brief Constructor.
  array_proxy_getter(native_type array): array_(array) {}

  /// @brief Return an array_proxy for a member array object.
  template <typename C>
  proxy_type operator()(C& c) { return make_array_proxy(c.*array_); }

  /// @brief Return an array_proxy for non-member array object.
  proxy_type operator()() { return make_array_proxy(*array_); }
private:
  native_type array_;
};

此仿函数的实例将包含在可调用的boost::python::object中。 make_array的单个入口点已展开:

/// @brief Create a callable Boost.Python object from an array.
template <typename Array>
boost::python::object make_array(Array array)
{ 
  // Deduce the array_proxy type by removing all the extents from the
  // array.
  ...

  // Register an array proxy.
  register_array_proxy<...>();

  // Create function.
  return boost::python::make_function(
      array_proxy_getter<Array>(array),
      ...);
}

最后,需要管理对象的生命周期。 Boost.Python提供了一些钩子来指定如何通过其CallPolices概念来管理对象生命周期。在这种情况下,with_custodian_and_ward_postcall可用于强制array_proxy<int>foo_vals()返回的foo延长了创建它的// CallPolicy type used to keep the owner alive when returning an object // that references the parents member variable. typedef boost::python::with_custodian_and_ward_postcall< 0, // return object (custodian) 1 // self or this (ward) > return_keeps_owner_alive; 实例的生命周期。

#include <string>
#include <typeinfo>
#include <boost/python.hpp>
#include <boost/python/suite/indexing/indexing_suite.hpp>

namespace detail {

template <typename> struct array_trait;

/// @brief Type that proxies to an array.
template <typename T>
class array_proxy
{
public:
  // Types
  typedef T           value_type;
  typedef T*          iterator;
  typedef T&          reference;
  typedef std::size_t size_type;

  /// @brief Empty constructor.
  array_proxy()
    : ptr_(0),
      length_(0)
  {}

  /// @brief Construct with iterators.
  template <typename Iterator>
  array_proxy(Iterator begin, Iterator end)
    : ptr_(&*begin),
      length_(std::distance(begin, end))
  {}

  /// @brief Construct with with start and size.
  array_proxy(reference begin, std::size_t length)
    : ptr_(&begin),
      length_(length)
  {}

  // Iterator support.
  iterator begin()               { return ptr_; }
  iterator end()                 { return ptr_ + length_; }

  // Element access.
  reference operator[](size_t i) { return ptr_[i]; }

  // Capacity.
  size_type size()               { return length_; }

private:
  T* ptr_;
  std::size_t length_;
};

/// @brief Make an array_proxy.
template <typename T>
array_proxy<typename array_trait<T>::element_type>
make_array_proxy(T& array)
{
  return array_proxy<typename array_trait<T>::element_type>(
    array[0],
    array_trait<T>::static_size);
}

/// @brief Policy type for referenced indexing, meeting the DerivedPolicies
///        requirement of boost::python::index_suite.
/// 
/// @note Requires Container to support:
///          - value_type and size_type types,
///          - value_type is default constructable and copyable,
///          - element access via operator[],
///          - Default constructable, iterator constructable,
///          - begin(), end(), and size() member functions
template <typename Container>
class ref_index_suite
  : public boost::python::indexing_suite<Container,
      ref_index_suite<Container> >
{
public:

  typedef typename Container::value_type data_type;
  typedef typename Container::size_type  index_type;
  typedef typename Container::size_type  size_type;

  // Element access and manipulation.

  /// @brief Get element from container.
  static data_type&
  get_item(Container& container, index_type index)
  {
    return container[index];
  }

  /// @brief Set element from container.
  static void
  set_item(Container& container, index_type index, const data_type& value)
  {
    container[index] = value;
  }

  /// @brief Reset index to default value.
  static void
  delete_item(Container& container, index_type index)
  {
    set_item(container, index, data_type());
  };

  // Slice support.

  /// @brief Get slice from container.
  ///
  /// @return Python object containing
  static boost::python::object
  get_slice(Container& container, index_type from, index_type to)
  {
    using boost::python::list;
    if (from > to) return list();

    // Return copy, as container only references its elements.
    list list;
    while (from != to) list.append(container[from++]);
    return list;
  };

  /// @brief Set a slice in container with a given value.
  static void
  set_slice(
    Container& container, index_type from,
    index_type to, const data_type& value
  )
  {
    // If range is invalid, return early.
    if (from > to) return;

    // Populate range with value.
    while (from < to) container[from++] = value;
  }

  /// @brief Set a slice in container with another range.
  template <class Iterator>
  static void
  set_slice(
    Container& container, index_type from,
    index_type to, Iterator first, Iterator last
  )
  {
    // If range is invalid, return early.
    if (from > to) return;

    // Populate range with other range.
    while (from < to) container[from++] = *first++;   
  }

  /// @brief Reset slice to default values.
  static void
  delete_slice(Container& container, index_type from, index_type to)
  {
    set_slice(container, from, to, data_type());
  }

  // Capacity.

  /// @brief Get size of container.
  static std::size_t
  size(Container& container) { return container.size(); }

  /// @brief Check if a value is within the container.
  template <class T>
  static bool
  contains(Container& container, const T& value)
  {
    return std::find(container.begin(), container.end(), value)
        != container.end();
  }

  /// @brief Minimum index supported for container.
  static index_type
  get_min_index(Container& /*container*/)
  {
      return 0;
  }

  /// @brief Maximum index supported for container.
  static index_type
  get_max_index(Container& container)
  {
    return size(container);
  }

  // Misc.

  /// @brief Convert python index (could be negative) to a valid container
  ///        index with proper boundary checks.
  static index_type
  convert_index(Container& container, PyObject* object)
  {
    namespace python = boost::python;
    python::extract<long> py_index(object);

    // If py_index cannot extract a long, then type the type is wrong so
    // set error and return early.
    if (!py_index.check()) 
    {
      PyErr_SetString(PyExc_TypeError, "Invalid index type");
      python::throw_error_already_set(); 
      return index_type();
    }

    // Extract index.
    long index = py_index();

    // Adjust negative index.
    if (index < 0)
        index += container.size();

    // Boundary check.
    if (index >= long(container.size()) || index < 0)
    {
      PyErr_SetString(PyExc_IndexError, "Index out of range");
      python::throw_error_already_set();
    }

    return index;
  }
};

/// @brief Trait for arrays.
template <typename T>
struct array_trait_impl;

// Specialize for native array.
template <typename T, std::size_t N>
struct array_trait_impl<T[N]>
{
  typedef T element_type;
  enum { static_size = N };
  typedef array_proxy<element_type> proxy_type;
  typedef boost::python::default_call_policies policy;
  typedef boost::mpl::vector<array_proxy<element_type> > signature;
};

// Specialize boost::array to use the native array trait.
template <typename T, std::size_t N>
struct array_trait_impl<boost::array<T, N> >
  : public array_trait_impl<T[N]>
{};

// @brief Specialize for member objects to use and modify non member traits.
template <typename T, typename C>
struct array_trait_impl<T (C::*)>
  : public array_trait_impl<T>
{
  typedef boost::python::with_custodian_and_ward_postcall<
      0, // return object (custodian)
      1  // self or this (ward)
    > policy;

  // Append the class to the signature.
  typedef typename boost::mpl::push_back<
    typename array_trait_impl<T>::signature, C&>::type signature;
};

/// @brief Trait class used to deduce array information, policies, and 
///        signatures
template <typename T>
struct array_trait:
  public array_trait_impl<typename boost::remove_pointer<T>::type>
{
  typedef T native_type;
};

/// @brief Functor used used convert an array to an array_proxy for
///        non-member objects.
template <typename Trait>
struct array_proxy_getter
{
public:
  typedef typename Trait::native_type native_type;
  typedef typename Trait::proxy_type proxy_type;

  /// @brief Constructor.
  array_proxy_getter(native_type array): array_(array) {}

  /// @brief Return an array_proxy for a member array object.
  template <typename C>
  proxy_type operator()(C& c) { return make_array_proxy(c.*array_); }

  /// @brief Return an array_proxy for a non-member array object.
  proxy_type operator()() { return make_array_proxy(*array_); }
private:
  native_type array_;
};

/// @brief Conditionally register a type with Boost.Python.
template <typename Trait>
void register_array_proxy()
{
  typedef typename Trait::element_type element_type;
  typedef typename Trait::proxy_type proxy_type;

  // If type is already registered, then return early.
  namespace python = boost::python;
  bool is_registered = (0 != python::converter::registry::query(
    python::type_id<proxy_type>())->to_python_target_type());
  if (is_registered) return;

  // Otherwise, register the type as an internal type.
  std::string type_name = std::string("_") + typeid(element_type).name();
  python::class_<proxy_type>(type_name.c_str(), python::no_init)
    .def(ref_index_suite<proxy_type>());
}

/// @brief Create a callable Boost.Python object that will return an
///        array_proxy type when called.
///
/// @note This function will conditionally register array_proxy types
///       for conversion within Boost.Python.  The array_proxy will
///       extend the life of the object from which it was called.
///       For example, if `foo` is an object, and `vars` is an array,
///       then the object returned from `foo.vars` will extend the life
///       of `foo`.
template <typename Array>
boost::python::object make_array_aux(Array array)
{
  typedef array_trait<Array> trait_type;
  // Register an array proxy.
  register_array_proxy<trait_type>();

  // Create function.
  return boost::python::make_function(
      array_proxy_getter<trait_type>(array),
      typename trait_type::policy(),
      typename trait_type::signature());
}

} // namespace detail

/// @brief Create a callable Boost.Python object from an array.
template <typename T>
boost::python::object make_array(T array)
{ 
  return detail::make_array_aux(array);
}

struct Foo
{
  int vals[3];
  boost::array<std::string, 5> strs;

  Foo()  { std::cout << "Foo()"  << std::endl; }
  ~Foo() { std::cout << "~Foo()" << std::endl; }
};

int more_vals[2];

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;

  python::class_<Foo>("Foo")
    .add_property("vals", make_array(&Foo::vals))
    .add_property("strs", make_array(&Foo::strs))
    ;
  python::def("more_vals", make_array(&more_vals));
}

以下是支持非成员和成员原生和Boost.Array单维数组的完整示例:

>>> from example import Foo, more_vals
>>> def print_list(l): print ', '.join(str(v) for v in l)
... 
>>> f = Foo()
Foo()
>>> f.vals[0] = 10
>>> print f.vals[0]
10
>>> f.vals[0] = '10'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Invalid assignment
>>> f.vals[100] = 10
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: Index out of range
>>> f.vals[:] = xrange(100,103)
>>> print_list(f.vals)
100, 101, 102
>>> f.strs[:] = ("a", "b", "c", "d", "e")
>>> print_list(f.strs)
a, b, c, d, e
>>> f.vals[-1] = 30
>>> print_list(f.vals)
100, 101, 30
>>> v = f.vals
>>> del v[:-1]
>>> print_list(f.vals)
0, 0, 30
>>> print_list(v)
0, 0, 30
>>> x = v[-1:]
>>> f = None
>>> v = None
~Foo()
>>> print_list(x)
30
>>> more_vals()[:] = xrange(50, 100)
>>> print_list(more_vals())
50, 51

使用,测试访问,切片,类型检查和生命周期管理:

{{1}}

答案 1 :(得分:3)

我认为它应该可以与boost {python的indexing suite一起使用。对于您的情况,您必须将容器指定为float*并在派生策略中返回常量。

编辑:上面的方法适用于容器,但很难用于您的情况。最简单的方法是声明两个函数set和get:

int getitem_foo(Foo & f, int index)
{
  if(index < 0 || index >=3)
  {
    PyErr_SetString(PyExc_IndexError, "index out of range");
    throw boost::python::error_already_set();;
  }
  return f.vals[index];
}

void setitem_foo(Foo& f, int index, int val)
{
  if(index < 0 || index >=3)
  {
    PyErr_SetString(PyExc_IndexError, "index out of range");
    throw boost::python::error_already_set();;
  }
  f.vals[index] = val;
}

然后:

boost::python::class_< Foo  >("Foo")
  .def("__getitem__", &getitem_foo)
  .def("__setitem__", &setitem_foo)
;

答案 2 :(得分:0)

我在上面尝试了@Tanner Sansbury的答案。它很有帮助,但有一些bug需要修复。在这里显示。

例如,有一个类似这样的类:

struct S {
    float a;
    float b;
    bool operator==(const S& s) {
        return a == s.a && b == s.b;
    }
};

class Foo {
 public:
    bool arr[100];
    S brr[100];
}

对于用户自定义类型,会发生编译错误。

首先,您应该实现自定义类型的operator ==函数,就像上面一样。

第二步,向array_proxy类添加更多代码。

/// @brief Type that proxies to an array.
template <typename T>
class array_proxy {
 public:
    // Types
    typedef std::ptrdiff_t difference_type;
    /// @brief Compare index supported for container.
    static bool
    compare_index(Container& container, index_type a, index_type b) {
        return a < b;
    }
}

然后,编译代码应该没问题。