c++ back end call the python level defined callbacks with swig wrapper

Question

I am wrapping a library which was written in C++ to Python API libwebqq

There is a type which is defined in boost function .

typedef boost::function&         


        

Answer 1

The problem seems to be that you haven't included any code to map from a Python callable to your EventListener class. It's not provided for free, although it's something that comes up fairly regularly, e.g. here which acted as a reference for part of this answer.

Your question has quite a lot of code that's not really relevant to the problem and not quite complete either so I've produced a minimal header file to demonstrate the problem and solution with:

#include <boost/function.hpp>
#include <string>
#include <map>

typedef boost::function<void (std::string)> EventListener;

enum QQEvent { THING };

inline std::map<QQEvent, EventListener>& table() {
  static std::map<QQEvent, EventListener> map;
  return map;
}

inline const EventListener& register_handler(const QQEvent& e, const EventListener& l) {
  return table()[e] = l;
}

inline void test(const QQEvent& e)  {
  table()[e]("Testing");
}

Given that header file a simple wrapper would be:

%module test

%{
#include "test.hh"
%}

%include "test.hh"

I also put together a bit of Python to run this with:

import test

def f(x):
  print(x)

test.register_handler(test.THING, f)
test.test(test.THING)

With this I can reproduce the error you see:

LD_LIBRARY_PATH=. python3.1 run.py
Traceback (most recent call last):
  File "run.py", line 6, in 
    test.register_handler(test.THING, f)
TypeError: in method 'register_handler', argument 2 of type 'EventListener const &'

Hopefully we're on the same page now. There's a single version of register_handler that expects an object of type EventListener (SWIG's generated proxy for the type to be precise). We're not trying to pass an EventListener in when we call that function though - it's a Python Callable instead, with not much known on the C++ side - certainly not a type match or implicitly convertible. So we need to add some glue in our interface to mush the Python type into the real C++ type.

We do that by defining an entirely new type, which only exists internally to the SWIG wrapper code (i.e. within %{ }%). The type PyCallback has one purpose: hold a reference to the real Python thing we're working with and make it look/feel like a function in C++.

Once we've added that PyCallback implementation detail (which nobody gets to see) we then need to add another overload for register_handler, which takes a PyObject* directly and constructs the PyCallback+EventListener for us. Since this only exists for the purpose of wrapping we use %inline to declare, define and wrap all within the SWIG interface file. So our interface file now looks like:

%module test

%{
#include "test.hh"

class PyCallback
{
    PyObject *func;
    PyCallback& operator=(const PyCallback&); // Not allowed
public:
    PyCallback(const PyCallback& o) : func(o.func) {
      Py_XINCREF(func);
    }
    PyCallback(PyObject *func) : func(func) {
      Py_XINCREF(this->func);
      assert(PyCallable_Check(this->func));
    }
    ~PyCallback() {
      Py_XDECREF(func);
    }
    void operator()(const std::string& s) {
      if (!func || Py_None == func || !PyCallable_Check(func))
        return;
      PyObject *args = Py_BuildValue("(s)", s.c_str());
      PyObject *result = PyObject_Call(func,args,0);
      Py_DECREF(args);
      Py_XDECREF(result);
    }
};
%}

%include "test.hh"

%inline %{
  void register_handler(const QQEvent& e, PyObject *callback) {
    register_handler(e, PyCallback(callback));
  }
%}

At this point we now have enough for the original test Python to run successfully.

It's worth noting that we could have chosen to hide the original overload of register_handler, but in this instance I prefer not to - it's more of a feature than a mistake to leave that visible because it permits you to manipulate C++ defined callbacks also, e.g. you could get/set them from the Python side and expose some core ones as global variables.

In your actual code you'll want to do:

%extend Action {
  void setCallback(PyObject *callback) {
    $self->setCallback(PyCallback(callback));
  }
}

to overload Action::setCallback, after the line %include "test.hh", instead of the %inline I used in my example to overload the free function.

Finally you might want to expose operator() of your Action class using %rename, or you could chose to expose the callback_ member using the function pointer/member function trick.