Track changes to lists and dictionaries in python?

Question

I have a class where the instances of this class needs to track the changes to its attributes.

Example: obj.att = 2 would be something that\'s easily tr

Answer 1

My jsonfile module detects changes of (nested) JSON compatible Python objects. Just subclass JSONFileRoot to adapt change detection for your needs.

>>> import jsonfile
>>> class Notify(jsonfile.JSONFileRoot):
...   def on_change(self):
...     print(f'notify: {self.data}')
... 
>>> test = Notify()
>>> test.data = 1
notify: 1
>>> test.data = [1,2,3]
notify: [1, 2, 3]
>>> test.data[0] = 12
notify: [12, 2, 3]
>>> test.data[1] = {"a":"b"}
notify: [12, {'a': 'b'}, 3]
>>> test.data[1]["a"] = 20
notify: [12, {'a': 20}, 3]

Note that it goes on the proxy class way how e-satis advised, without supporting sets.

Answer 2

I was curious how this might be accomplished when I saw the question, here is the solution I came up with. Not as simple as I would like it to be but it may be useful. First, here is the behavior:

class Tracker(object):
    def __init__(self):
        self.lst = trackable_type('lst', self, list)
        self.dct = trackable_type('dct', self, dict)
        self.revisions = {'lst': [], 'dct': []}


>>> obj = Tracker()            # create an instance of Tracker
>>> obj.lst.append(1)          # make some changes to list attribute
>>> obj.lst.extend([2, 3])
>>> obj.lst.pop()
3
>>> obj.dct['a'] = 5           # make some changes to dict attribute
>>> obj.dct.update({'b': 3})
>>> del obj.dct['a']
>>> obj.revisions              # check out revision history
{'lst': [[1], [1, 2, 3], [1, 2]], 'dct': [{'a': 5}, {'a': 5, 'b': 3}, {'b': 3}]}

Now the trackable_type() function that makes all of this possible:

def trackable_type(name, obj, base):
    def func_logger(func):
        def wrapped(self, *args, **kwargs):
            before = base(self)
            result = func(self, *args, **kwargs)
            after = base(self)
            if before != after:
                obj.revisions[name].append(after)
            return result
        return wrapped

    methods = (type(list.append), type(list.__setitem__))
    skip = set(['__iter__', '__len__', '__getattribute__'])
    class TrackableMeta(type):
        def __new__(cls, name, bases, dct):
            for attr in dir(base):
                if attr not in skip:
                    func = getattr(base, attr)
                    if isinstance(func, methods):
                        dct[attr] = func_logger(func)
            return type.__new__(cls, name, bases, dct)

    class TrackableObject(base):
        __metaclass__ = TrackableMeta

    return TrackableObject()

This basically uses a metaclass to override every method of an object to add some revision logging if the object changes. This is not super thoroughly tested and I haven't tried any other object types besides list and dict, but it seems to work okay for those.

Answer 3

You can also wrap the dictionary or list methods you want to track and make sure you do what you want inside the wrapper. Here is a dict example:

from functools import wraps

def _setChanged(func):
    @wraps(func)
    def wrappedFunc(self, *args, **kwargs):
        self.changed = True
        return func(self, *args, **kwargs)
    return wrappedFunc

def _trackObjectMethods(calssToTrack):
    for methodName in dir(calssToTrack):
        if methodName in calssToTrack._destructive:
            setattr(calssToTrack, methodName, _setChanged(getattr(calssToTrack, methodName)))

class Dictionary(dict):
    _destructive = ('__delitem__', '__setitem__', 'clear', 'pop', 'popitem', 'setdefault', 'update')

    def __init__(self, *args, **kwargs):
        self.changed = False
        super().__init__(*args, **kwargs)

_trackObjectMethods(Dictionary)

d = Dictionary()
print(d.changed)
d["1"] = 'test'
print(d.changed)
d.changed = False
print(d.changed)
d["12"] = 'test2'
print(d.changed)

As you can see if any items of the dictionary changes, the custom variable that I added to my custom Dictionary object will be set to True. This way I can tell if the object has changes since the last time I had set the changed variable to False.

Answer 4

You could take advantage of the abstract base classes in the collections module, which dict and list implement. This gives you a standard library interface to code against with a short list of methods to override, __getitem__, __setitem__, __delitem__, insert. Wrap the attributes in a trackable adapter inside __getattribute__.

import collections

class Trackable(object):
    def __getattribute__(self, name):
        attr = object.__getattribute__(self, name)
        if isinstance(attr, collections.MutableSequence):
            attr = TrackableSequence(self, attr)
        if isinstance(attr, collections.MutableMapping):
            attr = TrackableMapping(self, attr)
        return attr

    def __setattr__(self, name, value):
        object.__setattr__(self, name, value)
        # add change tracking


class TrackableSequence(collections.MutableSequence):
    def __init__(self, tracker, trackee):
        self.tracker = tracker
        self.trackee = trackee

    # override all MutableSequence's abstract methods
    # override the the mutator abstract methods to include change tracking


class TrackableMapping(collections.MutableMapping):
    def __init__(self, tracker, trackee):
        self.tracker = tracker
        self.trackee = trackee

    # override all MutableMapping's abstract methods
    # override the the mutator abstract methods to include change tracking

Answer 5

Instead of monkey patching, you can create a proxy class:

• Make a proxy class that inherit from dict/list/set whatever
• Intercept attribute setting, and if the value is a dict/list/set, wrap it into the proxy class
• In proxy class __getattribute__, make sure the method is called on the wrapped type, but take care of tracking before doing so.

Pro:

• no class alteration

Con:

• you are limited to a number of types you know and expect