Extending SWIG builtin classes

Question

The -builtin option of SWIG has the advantage of being faster, and of being exempt of a bug with multiple inheritance.
The setback is I can\'t set any attribute on the g

Answer 1

The problem comes from how swig implemented the classes in "-builtin" to be just like builtin classes (hence the name).

builtin classes are not extensible - try to add or modify a member of "str" and python won't let you modify the attribute dictionary.

I do have a solution I've been using for several years.

I'm not sure I can recommend it because:

1. It's arguably evil - the moral equivalent of casting away const-ness in C/C++
2. It's unsupported and could break in future python releases
3. I haven't tried it with python3
4. I would be a bit uncomfortable using "black-magic" like this in production code - it could break and is certainly obscure - but at least one giant corporation IS using this in production code

But.. I love how well it works to solve some obscure features we wanted for debugging.

The original idea is not mine, I got it from: https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader

The basic idea is to access the const dictionary in the swig-created type object as a non-const dictionary and add/override any desired methods.

FYI, the technique of runtime modification of classes is called monkeypatching, see https://en.wikipedia.org/wiki/Monkey_patch

First - here's "monkeypatch.py":

''' monkeypatch.py:
I got this from https://gist.github.com/mahmoudimus/295200 by Mahmoud Abdelkader,
his comment: "found this from Armin R. on Twitter, what a beautiful gem ;)"
I made a few changes for coding style preferences
- Rudy Albachten   April 30 2015
'''

import ctypes
from types import DictProxyType, MethodType

# figure out the size of _Py_ssize_t
_Py_ssize_t = ctypes.c_int64 if hasattr(ctypes.pythonapi, 'Py_InitModule4_64') else ctypes.c_int

# python without tracing
class _PyObject(ctypes.Structure):
    pass
_PyObject._fields_ = [
    ('ob_refcnt', _Py_ssize_t),
    ('ob_type', ctypes.POINTER(_PyObject))
]

# fixup for python with tracing
if object.__basicsize__ != ctypes.sizeof(_PyObject):
    class _PyObject(ctypes.Structure):
        pass
    _PyObject._fields_ = [
        ('_ob_next', ctypes.POINTER(_PyObject)),
        ('_ob_prev', ctypes.POINTER(_PyObject)),
        ('ob_refcnt', _Py_ssize_t),
        ('ob_type', ctypes.POINTER(_PyObject))
    ]

class _DictProxy(_PyObject):
    _fields_ = [('dict', ctypes.POINTER(_PyObject))]

def reveal_dict(proxy):
    if not isinstance(proxy, DictProxyType):
        raise TypeError('dictproxy expected')
    dp = _DictProxy.from_address(id(proxy))
    ns = {}
    ctypes.pythonapi.PyDict_SetItem(ctypes.py_object(ns), ctypes.py_object(None), dp.dict)
    return ns[None]

def get_class_dict(cls): 
    d = getattr(cls, '__dict__', None)
    if d is None:
        raise TypeError('given class does not have a dictionary')
    if isinstance(d, DictProxyType):
        return reveal_dict(d)
    return d

def test():
    import random
    d = get_class_dict(str)
    d['foo'] = lambda x: ''.join(random.choice((c.upper, c.lower))() for c in x)
    print "and this is monkey patching str".foo()

if __name__ == '__main__':
    test()

Here's a contrived example using monkeypatch:

I have a class "myclass" in module "mystuff" wrapped with swig -python -builtin

I want to add an extra runtime method "namelen" that returns the length of the name returned by myclass.getName()

import mystuff
import monkeypatch

# add a "namelen" method to all "myclass" objects
def namelen(self):
    return len(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen

x = mystuff.myclass("xxxxxxxx")
print "namelen:", x.namelen()

Note that this can also be used to extend or override methods on builtin python classes, as is demonstrated in the test in monkeypatch.py: it adds a method "foo" to the builtin str class that returns a copy of the original string with random upper/lower case letters

I would probably replace:

# add a "namelen" method to all "myclass" objects
def namelen(self):
    return len(self.getName())
d = monkeypatch.get_class_dict(mystuff.myclass)
d['namelen'] = namelen

with

# add a "namelen" method to all "myclass" objects
monkeypatch.get_class_dict(mystuff.myclass)['namelen'] = lambda self: return len(self.getName())

to avoid extra global variables

Answer 2

I found a solution quite by accident. I was experimenting with metaclasses, thinking I could manage to override the setattr and getattr functions of the builtin type in the subclass.

Doing this I discovered the builtins already have a metaclass (SwigPyObjectType), so my metaclass had to inherit it.

And that's it. This alone solved the problem. I would be glad if someone could explain why :

SwigPyObjectType = type(SWIGBuiltinClass)

class Meta(SwigPyObjectType):
    pass

class Thing(SWIGBuiltinClass):
    __metaclass__ = Meta

Thing.myattr = 'anything' # Works fine this time