Python: Can't pickle type X, attribute lookup failed

Question

I am trying to pickle a namedtuple:

from collections import namedtuple
import cPickle

class Foo:

    Bar = namedtuple(\'Bar\', [\'x\', \'y\'])         


        

Answer 1

Using dill in place of pickle here will allow this to work

Answer 2

Yes, the fact that it's a class member is a problem:

>>> class Foo():
...     Bar = namedtuple('Bar', ['x','y'])
...     def baz(self):
...         b = Foo.Bar(x=2, y=3)
...         print(type(b))
...
>>> a = Foo()
>>> a.baz()
<class '__main__.Bar'>

The problem is that when namedtuple() returns a type object, it isn't aware of the fact that it's being assigned to a class member - and thus, it tells the type object that its type name should be __main__.Bar, even though it should really be __main__.Foo.Bar.

Answer 3

Nesting classes makes pickle fail, since it relies on the path of the object inside your application to reconstruct it later.

The immediate solution is to not nest classes, i.e. move Bar definition to outside Foo. Code will work all the same.

But a better thing to do is to not use pickle at all to store data. Use some other serialization format, like json, or a database, like sqlite3.

You have just hit one of the many inconveniences of pickle, if you change your code, move things around, or sometimes make small structural changes, your data becomes unloadable.

Besides that, pickle has other disadvantages: It is slow, unsecure, python-only...

Answer 4

The solution here is to move your named tuple definition to the module level, then pickle works. A detailed answer is provided here:

How to pickle a namedtuple instance correctly