Returning objects to Python from C

Question

I've read the documentation for the Python C-API, and even written a few extension modules. However, I'm still a bit unclear on the exact semantics when it comes to returning Python objects from a C function.

The limited examples in the Python docs usually show a C function which returns the result of Py_BuildValue. Now, Py_BuildValue returns a New Reference, and transfers ownership of this reference over to the interpreter. So, can I extrapolate from this that it is a general rule that any object returned to Python must be a new reference, and that returning an object from a C function is the same as transferring ownership of the object over to the interpreter?

If so, what about cases where you return an object that is already owned by something? For example, suppose you write a C function which takes in a PyObject* which is a tuple, and you call PyTuple_GetItem on it and return the result. PyTuple_GetItem returns a borrowed reference - meaning that the item is still "owned" by the tuple. So, does a C function which returns the result of something like PyTuple_GetItem have to INCREF the result before returning it to the interpreter?

For example:

static PyObject* my_extension_module(PyObject* tup)
{
   PyObject* item = PyTuple_GetItem(tup, 1);
   if (!item) { /* handle error */ }

   return item; // <--- DO WE NEED TO INCREF BEFORE RETURNING HERE?
}

Answer 1

Python expects any function you expose to it to return a new reference, yes. If you only have a borrowed reference, you have to call Py_INCREF to give a new reference. If you return a borrowed reference, Python will proceed to call Py_DECREF (when it's done with the reference), and eventually that will cause the object to be freed while it is still in use.

(It's not uncommon for that "eventual" freeing to happen during interpreter exit, in which case it may go unnoticed, but it's still a mistake to return borrowed references.)