Python: how to implement __getattr__()?

Question

My class has a dict, for example:

class MyClass(object):
    def __init__(self):
        self.data = {\'a\': \'v1\', \'b\': \'v2\'}

Then I

Answer 1

I figured out an extension to @glglgl's answer that handles nested dictionaries and dictionaries insides lists that are in the original dictionary:

class d(dict):
    def __init__(self, *a, **k): 
        super(d, self).__init__(*a, **k)
        self.__dict__ = self
        for k in self.__dict__:
            if isinstance(self.__dict__[k], dict):
                self.__dict__[k] = d(self.__dict__[k])
            elif isinstance(self.__dict__[k], list):
                for i in range(len(self.__dict__[k])):
                    if isinstance(self.__dict__[k][i], dict):
                        self.__dict__[k][i] = d(self.__dict__[k][i])

Answer 2

class A(object):
  def __init__(self):
     self.data = {'a': 'v1', 'b': 'v2'}
  def __getattr__(self, attr):
     try:
       return self.data[attr]
     except:
       return "not found"


>>>a = A()
>>>print a.a
v1
>>>print a.c
not found

Answer 3

I think this implement is cooler

class MyClass(object):
    def __init__(self):
        self.data = {'a': 'v1', 'b': 'v2'}
    def __getattr__(self,key):
        return self.data.get(key,None)

Answer 4

class MyClass(object):

    def __init__(self):
        self.data = {'a': 'v1', 'b': 'v2'}

    def __getattr__(self, attr):
        return self.data[attr]

---

>>> ob = MyClass()
>>> v = ob.a
>>> v
'v1'

Be careful when implementing __setattr__ though, you will need to make a few modifications:

class MyClass(object):

    def __init__(self):
        # prevents infinite recursion from self.data = {'a': 'v1', 'b': 'v2'}
        # as now we have __setattr__, which will call __getattr__ when the line
        # self.data[k] tries to access self.data, won't find it in the instance 
        # dictionary and return self.data[k] will in turn call __getattr__
        # for the same reason and so on.... so we manually set data initially
        super(MyClass, self).__setattr__('data', {'a': 'v1', 'b': 'v2'})

    def __setattr__(self, k, v):
        self.data[k] = v

    def __getattr__(self, k):
        # we don't need a special call to super here because getattr is only 
        # called when an attribute is NOT found in the instance's dictionary
        try:
            return self.data[k]
        except KeyError:
            raise AttributeError

---

>>> ob = MyClass()
>>> ob.c = 1
>>> ob.c
1

If you don't need to set attributes just use a namedtuple eg.

>>> from collections import namedtuple
>>> MyClass = namedtuple("MyClass", ["a", "b"])
>>> ob = MyClass(a=1, b=2)
>>> ob.a
1

If you want the default arguments you can just write a wrapper class around it:

class MyClass(namedtuple("MyClass", ["a", "b"])):

    def __new__(cls, a="v1", b="v2"):
        return super(MyClass, cls).__new__(cls, a, b)

or maybe it looks nicer as a function:

def MyClass(a="v1", b="v2", cls=namedtuple("MyClass", ["a", "b"])):
    return cls(a, b)

---

>>> ob = MyClass()
>>> ob.a
'v1'

Answer 5

I like to take this therefore.

I took it from somewhere, but I don't remember where.

class A(dict):
    def __init__(self, *a, **k):
        super(A, self).__init__(*a, **k)
        self.__dict__ = self

This makes the __dict__ of the object the same as itself, so that attribute and item access map to the same dict:

a = A()
a['a'] = 2
a.b = 5
print a.a, a['b'] # prints 2 5

Answer 6

A simple approach to solving your __getattr__()/__setattr__() infinite recursion woes

Implementing one or the other of these magic methods can usually be easy. But when overriding them both, it becomes trickier. This post's examples apply mostly to this more difficult case.

When implementing both these magic methods, it's not uncommon to get stuck figuring out a strategy to get around recursion in the __init__() constructor of classes. This is because variables need to be initialized for the object, but every attempt to read or write those variables go through __get/set/attr__(), which could have more unset variables in them, incurring more futile recursive calls.

Up front, a key point to remember is that __getattr__() only gets called by the runtime if the attribute can't be found on the object already. The trouble is to get attributes defined without tripping these functions recursively.

Another point is __setattr__() will get called no matter what. That's an important distinction between the two functions, which is why implementing both attribute methods can be tricky.

This is one basic pattern that solves the problem.

class AnObjectProxy:
    _initialized = False # *Class* variable 'constant'.

    def __init__(self):
        self._any_var = "Able to access instance vars like usual."
        self._initialized = True # *instance* variable.

    def __getattr__(self, item):
        if self._initialized:
            pass # Provide the caller attributes in whatever ways interest you.
        else:
            try:
                return self.__dict__[item] # Transparent access to instance vars.
            except KeyError:
                raise AttributeError(item)

    def __setattr__(self, key, value):
        if self._initialized:
            pass # Provide caller ways to set attributes in whatever ways.
        else:
            self.__dict__[key] = value # Transparent access.

While the class is initializing and creating it's instance vars, the code in both attribute functions permits access to the object's attributes via the __dict__ dictionary transparently - your code in __init__() can create and access instance attributes normally. When the attribute methods are called, they only access self.__dict__ which is already defined, thus avoiding recursive calls.

In the case of self._any_var, once it's assigned, __get/set/attr__() won't be called to find it again.

Stripped of extra code, these are the two pieces that are most important.

...     def __getattr__(self, item):
...         try:
...             return self.__dict__[item]
...         except KeyError:
...             raise AttributeError(item)
... 
...     def __setattr__(self, key, value):
...         self.__dict__[key] = value

Solutions can build around these lines accessing the __dict__ dictionary. To implement an object proxy, two modes were implemented: initialization and post-initialization in the code before this - a more detailed example of the same is below.

There are other examples in answers that may have differing levels of effectiveness in dealing with all aspects of recursion. One effective approach is accessing __dict__ directly in __init__() and other places that need early access to instance vars. This works but can be a little verbose. For instance,

self.__dict__['_any_var'] = "Setting..."

would work in __init__().

My posts tend to get a little long-winded.. after this point is just extra. You should already have the idea with the examples above.

A drawback to some other approaches can be seen with debuggers in IDE's. They can be overzealous in their use of introspection and produce warning and error recovery messages as you're stepping through code. You can see this happening even with solutions that work fine standalone. When I say all aspects of recursion, this is what I'm talking about.

The examples in this post only use a single class variable to support 2-modes of operation, which is very maintainable.

But please NOTE: the proxy class required two modes of operation to set up and proxy for an internal object. You don't have to have two modes of operation.

You could simply incorporate the code to access the __dict__ as in these examples in whatever ways suit you.

If your requirements don't include two modes of operation, you may not need to declare any class variables at all. Just take the basic pattern and customize it.

Here's a closer to real-world (but by no means complete) example of a 2-mode proxy that follows the pattern:

>>> class AnObjectProxy:
...     _initialized = False  # This class var is important. It is always False.
...                           # The instances will override this with their own, 
...                           # set to True.
...     def __init__(self, obj):
...         # Because __getattr__ and __setattr__ access __dict__, we can
...         # Initialize instance vars without infinite recursion, and 
...         # refer to them normally.
...         self._obj         = obj
...         self._foo         = 123
...         self._bar         = 567
...         
...         # This instance var overrides the class var.
...         self._initialized = True
... 
...     def __setattr__(self, key, value):
...         if self._initialized:
...             setattr(self._obj, key, value) # Proxying call to wrapped obj.
...         else:
...             # this block facilitates setting vars in __init__().
...             self.__dict__[key] = value
... 
...     def __getattr__(self, item):
...         if self._initialized:
...             attr = getattr(self._obj, item) # Proxying.
...             return attr
...         else:
...             try:
...                 # this block facilitates getting vars in __init__().
...                 return self.__dict__[item]
...             except KeyError:
...                 raise AttributeError(item)
... 
...     def __call__(self, *args, **kwargs):
...         return self._obj(*args, **kwargs)
... 
...     def __dir__(self):
...         return dir(self._obj) + list(self.__dict__.keys())

The 2-mode proxy only needs a bit of "bootstrapping" to access vars in its own scope at initialization before any of its vars are set. After initialization, the proxy has no reason to create more vars for itself, so it will fare fine by deferring all attribute calls to it's wrapped object.

Any attribute the proxy itself owns will still be accessible to itself and other callers since the magic attribute functions only get called if an attribute can't be found immediately on the object.

Hopefully this approach can be of benefit to anyone who appreciates a direct approach to resolving their __get/set/attr__() __init__() frustrations.