Calling C/C++ from Python? [closed]

Answer 1

pybind11 minimal runnable example

pybind11 was previously mentioned at https://stackoverflow.com/a/38542539/895245 but I would like to give here a concrete usage example and some further discussion about implementation.

All and all, I highly recommend pybind11 because it is really easy to use: you just include a header and then pybind11 uses template magic to inspect the C++ class you want to expose to Python and does that transparently.

The downside of this template magic is that it slows down compilation immediately adding a few seconds to any file that uses pybind11, see for example the investigation done on this issue. PyTorch agrees. A proposal to remediate this problem has been made at: https://github.com/pybind/pybind11/pull/2445

Here is a minimal runnable example to give you a feel of how awesome pybind11 is:

class_test.cpp

#include 

#include 

struct ClassTest {
    ClassTest(const std::string &name) : name(name) { }
    void setName(const std::string &name_) { name = name_; }
    const std::string &getName() const { return name; }
    std::string name;
};

namespace py = pybind11;

PYBIND11_PLUGIN(class_test) {
    py::module m("my_module", "pybind11 example plugin");
    py::class_(m, "ClassTest")
        .def(py::init())
        .def("setName", &ClassTest::setName)
        .def("getName", &ClassTest::getName)
        .def_readwrite("name", &ClassTest::name);
    return m.ptr();
}

class_test_main.py

#!/usr/bin/env python3

import class_test

my_class_test = class_test.ClassTest("abc");
print(my_class_test.getName())
my_class_test.setName("012")
print(my_class_test.getName())
assert(my_class_test.getName() == my_class_test.name)

Compile and run:

#!/usr/bin/env bash
set -eux
g++ `python3-config --cflags` -shared -std=c++11 -fPIC class_test.cpp \
  -o class_test`python3-config --extension-suffix` `python3-config --libs`
./class_test_main.py

This example shows how pybind11 allows you to effortlessly expose the ClassTest C++ class to Python! Compilation produces a file named class_test.cpython-36m-x86_64-linux-gnu.so which class_test_main.py automatically picks up as the definition point for the class_test natively defined module.

Perhaps the realization of how awesome this is only sinks in if you try to do the same thing by hand with the native Python API, see for example this example of doing that, which has about 10x more code: https://github.com/cirosantilli/python-cheat/blob/4f676f62e87810582ad53b2fb426b74eae52aad5/py_from_c/pure.c On that example you can see how the C code has to painfully and explicitly define the Python class bit by bit with all the information it contains (members, methods, further metadata...). See also:

• Can python-C++ extension get a C++ object and call its member function?
• Exposing a C++ class instance to a python embedded interpreter
• A full and minimal example for a class (not method) with Python C Extension?
• Embedding Python in C++ and calling methods from the C++ code with Boost.Python
• Inheritance in Python C++ extension

pybind11 claims to be similar to Boost.Python which was mentioned at https://stackoverflow.com/a/145436/895245 but more minimal because it is freed from the bloat of being inside the Boost project:

pybind11 is a lightweight header-only library that exposes C++ types in Python and vice versa, mainly to create Python bindings of existing C++ code. Its goals and syntax are similar to the excellent Boost.Python library by David Abrahams: to minimize boilerplate code in traditional extension modules by inferring type information using compile-time introspection.

The main issue with Boost.Python—and the reason for creating such a similar project—is Boost. Boost is an enormously large and complex suite of utility libraries that works with almost every C++ compiler in existence. This compatibility has its cost: arcane template tricks and workarounds are necessary to support the oldest and buggiest of compiler specimens. Now that C++11-compatible compilers are widely available, this heavy machinery has become an excessively large and unnecessary dependency.

Think of this library as a tiny self-contained version of Boost.Python with everything stripped away that isn't relevant for binding generation. Without comments, the core header files only require ~4K lines of code and depend on Python (2.7 or 3.x, or PyPy2.7 >= 5.7) and the C++ standard library. This compact implementation was possible thanks to some of the new C++11 language features (specifically: tuples, lambda functions and variadic templates). Since its creation, this library has grown beyond Boost.Python in many ways, leading to dramatically simpler binding code in many common situations.

pybind11 is also the only non-native alternative hightlighted by the current Microsoft Python C binding documentation at: https://docs.microsoft.com/en-us/visualstudio/python/working-with-c-cpp-python-in-visual-studio?view=vs-2019 (archive).

Tested on Ubuntu 18.04, pybind11 2.0.1, Python 3.6.8, GCC 7.4.0.