Python - anyone have a memoizing decorator that can handle unhashable arguments?

Question

I\'ve been using the following memoizing decorator (from the great book Python Algorithms: Mastering Basic Algorithms in the Python Language ... love it, btw).

Answer 1

Not heavily tested, but seems to work:

from functools import wraps

def memo(func):
    cache = []
    argslist = []
    @ wraps(func)
    def wrap(*args):
        try:
            result = cache[argslist.index(args)]
            print 'cache hit'
            return result
        except ValueError:
            argslist.append(args)
            cache.append(func(*args))
            print 'cache miss'
            return cache[-1]
    return wrap

d1 = { 'a':3, 'b':42 }
d2 = { 'c':7, 'd':19 }
d3 = { 'e':34, 'f':67 }

@memo
def func(d):
    return sum(d.values())

print func(d1)
# cache miss
# 45
print func(d2)
# cache miss
# 26
print func(d3)
# cache miss
# 101
print func(d2)
# cache hit
# 26

Answer 2

Technically you can solve this problem by turning the dict (or list or set) into a tuple. For example:

 key = tuple(the_dict.iteritems())
 key = tuple(the_list)
 key = tuple(sorted(the_set))

 cache[key] = func( .. )

But I wouldn't do this in memo, I'd rather change the functions you want to use memo on - for example, instead of accepting a dict they should only accept (key, value) pairs, instead of taking lists or sets they should just take *args.

Answer 3

Here is the example in Alex Martelli Python Cookbook that show how to create a memoize decorator using cPickle for function that take mutable argument (original version) :

import cPickle

class MemoizeMutable:
    def __init__(self, fn):
        self.fn = fn
        self.memo = {}
    def __call__(self, *args, **kwds):
        import cPickle
        str = cPickle.dumps(args, 1)+cPickle.dumps(kwds, 1)
        if not self.memo.has_key(str): 
            print "miss"  # DEBUG INFO
            self.memo[str] = self.fn(*args, **kwds)
        else:
            print "hit"  # DEBUG INFO

        return self.memo[str]

Here is a link.

EDIT: Using the code that you have given and this memoize decorator :

_level = MemoizeMutable(_level)

equirements = {'a': [],
               'b': [],
               'c': ['a'],
               'd': ['a','b'],
               'e': ['c','d'],
               'f': ['e']
                 }

print uses_hierarchy(equirements)

i was able to reproduce this:

miss
miss
hit
miss
miss
hit
miss
hit
hit
hit
miss
hit
{0: ['a', 'b'], 1: ['c', 'd'], 2: ['e'], 3: ['f']}

Answer 4

I've searched and have found a good python package.

https://pypi.org/project/memoization/

• easy to use
• possible to use unhashable arguments
• other useful option (time to live)

Answer 5

Since no one else has mentioned it, the Python Wiki has a Decorator Library which includes a number of memoizing decorator patterns.

My personal preference is the last one, which lets calling code simply treat the method as a lazily-evaluated property, rather than a method. But I like the implementation here better.

class lazy_property(object):
    '''Decorator: Enables the value of a property to be lazy-loaded.
    From Mercurial's util.propertycache

    Apply this decorator to a no-argument method of a class and you
    will be able to access the result as a lazy-loaded class property.
    The method becomes inaccessible, and the property isn't loaded
    until the first time it's called.  Repeated calls to the property
    don't re-run the function.

    This takes advantage of the override behavior of Descriptors - 
    __get__ is only called if an attribute with the same name does
    not exist.  By not setting __set__ this is a non-data descriptor,
    and "If an instance's dictionary has an entry with the same name
    as a non-data descriptor, the dictionary entry takes precedence."
     - http://users.rcn.com/python/download/Descriptor.htm

    To trigger a re-computation, 'del' the property - the value, not
    this class, will be deleted, and the value will be restored upon
    the next attempt to access the property.
    '''
    def __init__(self,func):
        self.func = func
        self.name = func.__name__
    def __get__(self, obj, type=None):
        result = self.func(obj)
        setattr(obj, self.name, result)
        return result

In the same file linked above there's also a lazy_dict decorator, which lets you treat a function as a dictionary with lazily-evaluated values.