Passing list of numpy arrays to C using cython

Question

I have a list list_of_arrays of 3D numpy arrays that I want to pass to a C function with the template

int my_func_c(double **data, int **shape,          


        

Answer 1

I think this is a good pattern to consume C-functionality from C++-code, and it can be also used here and would have two advantages:

1. Memory management is taken care of.
2. Thanks to templates, no casting needed, so we still have the safety-net of c's type safety.

To solve your problems you could use std::vector:

import numpy as np
cimport numpy as np
from libcpp.vector cimport vector

cdef extern from "my_func.c":
    double my_func_c(double **data, int **shape, int n_arrays)

def my_func(list list_of_arrays):
    cdef int n_arrays  = len(list_of_arrays)
    cdef vector[double *] data
    cdef vector [vector[int]] shape_mem # for storing casted shapes
    cdef vector[int *] shape  #pointers to stored shapes
    cdef double x
    cdef np.ndarray[double, ndim=3, mode="c"] temp

    shape_mem.resize(n_arrays)  
    for i in range(n_arrays):
        print "i:", i
        temp = list_of_arrays[i]
        data.push_back(&temp[0,0,0])
        for j in range(3):
            shape_mem[i].push_back(temp.shape[j])
        shape.push_back(shape_mem[i].data())

    x = my_func_c(data.data(), shape.data(), n_arrays)

    return x

Also your setup would need a modification:

# setup.py    
from distutils.core import setup, Extension
from Cython.Build import cythonize
import numpy as np

setup(ext_modules=cythonize(Extension(
            name='my_func_c',
            language='c++',
            extra_compile_args=['-std=c++11'],
            sources = ["my_func_c.pyx", "my_func.c"],
            include_dirs=[np.get_include()]
    )))

I prefer to use std::vector.data() over &data[0] because the second would mean undefined behavior for empty data, and that is the reason we need std=c++11 flag.

But in the end, it is for you to decide, which trade-off to make: the additional complexity of C++ (it has it own pitfalls) vs. handmade memory management vs. letting go of type safety for a short moment.