Where should I put Py_INCREF and Py_DECREF on this block in Python C Extension?

Question

Whenever I called my function, memory usage is increased around +10M per call, so I think there is some memory leak here.

....
PyObject *pair = PyTuple_New(2         


        

Answer 1

When an object is created, its refcount will be 1, so after this:

my_item  = PyInt_FromLong(jp)

the object at my_item will have a refcount of 1.

When you store an item into a container, the item's reference count is incremented so that the item will be retained, so after this:

PyList_Append(my_list, my_item);

the object at my_item will have a refcount of 2.

Therefore, this line:

PyList_Append(item, PyInt_FromLong(jp));

will create an object with refcount 1 and store it in the list, incrementing the object's refcount to 2.

Answer 2

The objects you create with PyInt_FromLong() and you add to the list should be kept in a local variable.

The reason are the ownership rules: PyInt_FromLong() generates a reference that you own. In the call to PyTuple_SetItem(), you lose this ownership again, because PyTuple_SetItem() "steals" it from you, so you don't have to care about. But PyList_Append() doesn't do so, it increases the refcount. In order to have the object GC'ed correctly, you have to release your ownership by DECREF'ing.

So, instead of PyList_Append(item, PyInt_FromLong(jp)), you do the following:

PyObject * jpo = PyInt_FromLong(jp);
// do some error checking here
PyList_Append(item, jpo);
Py_DECREF(jpo);

This will make the program do the right thing.