PEP 3118 warning when using ctypes array as numpy array

Question

I'm getting the following warning message when I try to use a ctypes array as a numpy array:

Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes, numpy
>>> TenByteBuffer = ctypes.c_ubyte * 10
>>> a = TenByteBuffer()
>>> b = numpy.ctypeslib.as_array(a)
C:\Python27\lib\site-packages\numpy\ctypeslib.py:402: RuntimeWarning: Item size
computed from the PEP 3118 buffer format string does not match the actual item s
ize.
  return array(obj, copy=False)
>>> b
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=uint8)

The code seems to be working, though. Is it a bad idea to ignore this warning?

Background: I'm calling a C DLL that produces data in real time. I need to pass the DLL a series of buffers to hold the data. While waiting for the next buffer to fill, I'd like to process the most recent buffer with numpy and save the result. I'm generating the buffers with code like above, and things seem to be working, but I don't want to sweep an important problem under the rug.

Answer 1

It's a bug in Python. ctypes currently produces invalid PEP 3118 type codes, which Numpy notices: http://bugs.python.org/issue10746 http://bugs.python.org/issue10744

When such an inconsistency is present, Numpy skips using the PEP 3118 buffer interface, and falls back to the old (obsolete) buffer interface. This should work properly.

You can silence the warning using Python's warnings module. However, the warning may have a performance impact.

You can also try working around the issue by wrapping the ctypes object in buffer().

Answer 2

There is a more convenient way of doing this, which avoids the warning altogether:

Instead of creating the data as a ctypes array first and then converting it to a NumPy array, just create it as a NumPy array right away, and then use numpy.ctypeslib.ndpointer as type specifier in your ctypes prototype. As an example, let's say you have a C function called f which takes a char* and a size_t as arguments:

void f(char* buf, size_t len);

Your ctypes prototype would be

from numpy.ctypeslib import ndpointer
some_dll = ctypes.CDLL(...)
some_dll.f.argtypes = [ndpointer(numpy.uint8, flags="C_CONTIGUOUS"),
                       ctypes.c_size_t]
some_dll.f.restype = None

and you can call this function as

a = numpy.zeros(10, numpy.uint8)
f(a, a.size)