Using descriptors in unhashable classes - python

Question

A common design pattern when using python descriptors is to have the descriptor keep a dictionary of instances using that descriptor. For example, suppose I want to make an attribute that counts the number of times it's accessed:

class CountingAttribute(object):

    def __init__(self):
        self.count = 0
        self.value = None


class MyDescriptor(object):

    def __init__(self):
        self.instances = {} #instance -> CountingAttribute

    def __get__(self, inst, cls):
        if inst in self.instances:
           ca = self.instances[inst]
        else:
            ca = CountingAttribute()
            self.instances[inst] = ca
        ca.count += 1
        return ca


class Foo(object):
    x = MyDescriptor()


def main():
    f = Foo()
    f.x
    f.x
    print("f.x has been accessed %d times (including the one in this print)"%(f.x.count,))

if __name__ == "__main__":
    main()

This is a completely silly example that doesn't do anything useful; I'm trying to isolate the main point.

The problem is that I can't use this descriptor in a class which isn't hashable, because the line

self.instances[inst] = ca

uses instances as a dictionary key. Is there a wise way of handling this sort of case? For example, one immediately thinks to use the instance's id, but I'm not sure if doing that will break something about how hashes are supposed to be used.

EDIT: I realize that instances should be something like a weakref.WeakKeyDictionary but I'm trying to keep it simple here to focus on the issue of hashability.

Answer 1

You could use id(inst) as a key.

Be aware that this doesn't cover the case that an object is destroyed and a new one is created with a new id.

In order to detect this properly, you should store the ca and a weakref in the dictionary. If you detect that the weakref's referred object is gone, you have to assume that the given id is reused.

Something like

import weakref

class MyDescriptor(object):

    def __init__(self):
        self.instances = {} #instance -> CountingAttribute

    def __get__(self, inst, cls):
        if inst is None: return self.instances # operating on the class, we get the dictionary.
        i = id(inst)
        if i in self.instances:
            ca, wr = self.instances[i]
            if wr() is None: del self.instances[i]
        if i not in self.instances:
            ca = CountingAttribute()
            self.instances[i] = (ca, weakref.ref(inst))
        ca.count += 1
        return ca

This relieves from the hashability problems conntected to a WeakKeyDictionary.

But maybe you don't need the dict at all. A completely different approach could be

class MyDescriptor(object):

    def __get__(self, inst, cls):
        if inst is None: return self, cls
        try:
            ca = inst.__the_ca
        except AttributeError:
            ca = inst.__the_ca = CountingAttribute()
        ca.count += 1
        return ca

This approach has its downsides as well. For example, you cannot easily use the descriptor more than once in a class without making it ugly as well. Thus, it should only be used with care. The first solution is, while more complex, the most uncomplicated one.