Wrap C struct with array member for access in python: SWIG? cython? ctypes?

Question

I want to access a C function that returns a struct containing double arrays (where the lengths of these arrays is given by other int members of the struct) from python. The dec

Answer 1

You could always write helper functions that take an "element *" and return the element you seek:

double element_get_weight(const element *elt, unsigned n) {
    assert(n < elt->ngi);  /* or similar */
    return elt->weight[n];
}

If you need to modify as well as read, you will want separate "getters" and "setters", of course.

SWIG should be able to wrap all of these easily and expose them to Python.

Performance might not be great, but probably no worse than the alternatives.

Answer 2

An equivalent to the SWIG created module using ctypes looks as follows:

from ctypes import *
from numpy import *

lib = cdll.LoadLibrary("_get_element.so")

class ELEMENT(Structure):
    _fields_ = [("dim", c_int),
                ("vertices", c_int),
                ("quadrature_degree", c_int),
                ("polynomial_degree", c_int),
                ("ngi", c_int),
                ("quadrature_familiy", c_int),
                ("weight", POINTER(c_double)),
                ("l", POINTER(c_double)),
                ("n", POINTER(c_double)),
                ("dn", POINTER(c_double))]

cget_element = lib.get_element
cget_element.argtypes = [c_int, c_int, c_int, c_int, POINTER(ELEMENT)]
cget_element.restype = None

def get_element(dim, vertices, quad_degree, poly_degree):
    e = ELEMENT()
    cget_element(dim, vertices, quad_degree, poly_degree, byref(e))
    weight = asarray([e.weight[i] for i in xrange(e.ngi)], dtype=float64)
    l = asarray([e.l[i] for i in xrange(e.ngi*e.dim)], dtype=float64).reshape((e.ngi,e.dim))
    n = asarray([e.n[i] for i in xrange(e.ngi*e.vertices)], dtype=float64).reshape((e.ngi,e.vertices))
    dn = asarray([e.dn[i] for i in xrange(e.ngi*e.vertices*e.dim)], dtype=float64).reshape((e.ngi,e.vertices,e.dim))
    return weight, l, n, dn

Answer 3

Using SWIG requires a typemap for the entire struct. Tyepmaps for only the pointer members are not enough, since they don't have the context to know what size to initialize the NumPy arrays with. I managed to get what I wanted with the following typemaps (which was basically copy & paste from numpy.i and adapt to my needs, probably not very robust):

%typemap (in,numinputs=0) element * (element temp) {
  $1 = &temp;
}

%typemap (argout) element * {
  /* weight */
  {
    npy_intp dims[1] = { $1->ngi };
    PyObject * array = PyArray_SimpleNewFromData(1, dims, NPY_DOUBLE, (void*)($1->weight));
    if (!array) SWIG_fail;
    $result = SWIG_Python_AppendOutput($result,array);
  }
  /* l */
  {
    npy_intp dims[2] = { $1->ngi, $1->dim };
    PyObject * array = PyArray_SimpleNewFromData(2, dims, NPY_DOUBLE, (void*)($1->l));
    if (!array) SWIG_fail;
    $result = SWIG_Python_AppendOutput($result,array);
  }
  /* n */
  {
    npy_intp dims[2] = { $1->ngi, $1->vertices };
    PyObject * array = PyArray_SimpleNewFromData(2, dims, NPY_DOUBLE, (void*)($1->n));
    if (!array) SWIG_fail;
    $result = SWIG_Python_AppendOutput($result,array);
  }
  /* dn */
  {
    npy_intp dims[3] = { $1->ngi, $1->vertices, $1->dim };
    PyObject * array = PyArray_SimpleNewFromData(3, dims, NPY_DOUBLE, (void*)($1->dn));
    if (!array) SWIG_fail;
    $result = SWIG_Python_AppendOutput($result,array);
  }
}

This works different from the C function in that it returns a tuple of NumPy arrays with the data I want, which is more convenient than having to extract it from the element object later. The first typemap furthermore eliminates the need to pass in an object of type element. Hence I can hide the element struct entirely from the python user.

The python interface finally looks like this:

weight, l, n, dn = get_element(dim, vertices, quadrature_degree, polynomial_degree)

Answer 4

Cython rules:

cdef extern from "the header.h":

ctypedef struct element:
  int dim
  int vertices
  int quadrature_degree
  int polynomial_degree
  int ngi
  int quadrature_familiy
  double *weight
  double *l
  double *n
  double *dn

void get_element(int dim, int vertices, int quad_degree, int poly_degree, element* e)

and then you can interface it, from python space

Answer 5

Check out SWIG's typemaps. They let you write your own code for handling specific types, specific instances (type+name) or even groups of arguments. I haven't done it for structures, but to specially handle a case where the C function takes an array and its size:

%typemap(in) (int argc, Descriptor* argv) {
    /* Check if is a list */
    if (PyList_Check($input)) {
        int size = PyList_Size($input);
        $1 = size;
        ...
        $2 = ...;
    }
}

That will take the pair of arguments int argc, Descriptor* argv (since the names are provided they have to match as well) and pass you the PyObject used and you write whatever C code you need to do the conversion. You could do a typemap for double *dn that would use the NumPy C API to do the conversion.