Overriding append method after inheriting from a Python List

Question

I want to create a list that can only accept certain types. As such, I\'m trying to inherit from a list in Python, and overriding the append() method like so:

Answer 1

I have made some changes to your class. This seems to be working.

A couple of suggestions: don't use type as a keyword - type is a built in function. Python instance variables are accessed using the self. prefix. So use self.<variable name>.

class TypedList(list):
    def __init__(self, type):
        self.type = type

    def append(self, item):
        if not isinstance(item, self.type):
            raise TypeError, 'item is not of type %s' % self.type
        super(TypedList, self).append(item)  #append the item to itself (the list)

from types import *
tl = TypedList(StringType)
tl.append('abc')
tl.append(None)
Traceback (most recent call last):
  File "<pyshell#25>", line 1, in <module>
    tl.append(None)
  File "<pyshell#22>", line 7, in append
    raise TypeError, 'item is not of type %s' % self.type
TypeError: item is not of type <type 'str'>

Answer 2

instead of self.append(item) use super(TypedList, self).append(item) (see http://docs.python.org/library/functions.html#super)

Answer 3

Although I like @Alex Martini's answer, it's unfortunately not practical for most situations. There are WAY too many things both in the standard library and in other library code that expect list types to inherit from list. You can't even do json.dumps() without inheriting from list unless you write your own serialize method for your type.

I would disagree with @msw because you might want to expose something like this in a library and not have control over the other code.

The list class unfortunately doesn't let you override a single function like in Alex's example. The following overrides all functions that could potentially add to the list (see here for all list functions: https://docs.python.org/2/library/stdtypes.html#mutable-sequence-types).

import collections

class CheckedList(list, collections.MutableSequence):
    def __init__(self, check_method, iterator_arg=None):
        self.__check_method = check_method
        if not iterator_arg is None:
            self.extend(iterator_arg) # This validates the arguments...

    def insert(self, i, v):
        return super(CheckedList, self).insert(i, self.__check_method(v))

    def append(self, v):
        return super(CheckedList, self).append(self.__check_method(v))

    def extend(self, t):
        return super(CheckedList, self).extend([ self.__check_method(v) for v in t ])

    def __add__(self, t): # This is for something like `CheckedList(validator, [1, 2, 3]) + list([4, 5, 6])`...
        return super(CheckedList, self).__add__([ self.__check_method(v) for v in t ])

    def __iadd__(self, t): # This is for something like `l = CheckedList(validator); l += [1, 2, 3]`
        return super(CheckedList, self).__iadd__([ self.__check_method(v) for v in t ])

    def __setitem__(self, i, v):
        if isinstance(i, slice):
            return super(CheckedList, self).__setitem__(i, [ self.__check_method(v1) for v1 in v ]) # Extended slice...
        else:
            return super(CheckedList, self).__setitem__(i, self.__check_method(v))

    def __setslice__(self, i, j, t): # NOTE: extended slices use __setitem__, passing in a tuple for i
        return super(CheckedList, self).__setslice__(i, j, [ self.__check_method(v) for v in t ])

Answer 4

By not doing it in the first place.

If you don't want something of type X in your list, why are you putting it there?

This is not a sarcastic response. Adding type restrictions as you are trying is either unnecessary or affirmatively counter-productive. It is however, a common request from people coming from a language background which has strict compile-time type checking.

For the same reason you wouldn't attempt 'a string' / 2.0, you have the same control over what gets put in a list. Since a list will happily contain heterogeneous types, at best TypedList will move a run-time TypeError from the point where you use the item forward in time to where you append it to the list. Given Python's duck-typing explicitly checking isinstance precludes later expansion of the list to contain non-type instances while providing no benefit.

added OrderedDict information:

Per request in comment, assuming Python 2.7 or greater collections.OrderedDict will do it. Per that documentation page, given 2.4 or greater you have to add one.

Answer 5

I want to create a list that can only accept certain types. As such, I'm trying to inherit from a list in Python

Not the best approach! Python lists have so many mutating methods that you'd have to be overriding a bunch (and would probably forget some).

Rather, wrap a list, inherit from collections.MutableSequence, and add your checks at the very few "choke point" methods on which MutableSequence relies to implement all others.

import collections

class TypedList(collections.MutableSequence):

    def __init__(self, oktypes, *args):
        self.oktypes = oktypes
        self.list = list()
        self.extend(list(args))

    def check(self, v):
        if not isinstance(v, self.oktypes):
            raise TypeError, v

    def __len__(self): return len(self.list)

    def __getitem__(self, i): return self.list[i]

    def __delitem__(self, i): del self.list[i]

    def __setitem__(self, i, v):
        self.check(v)
        self.list[i] = v

    def insert(self, i, v):
        self.check(v)
        self.list.insert(i, v)

    def __str__(self):
        return str(self.list)

The oktypes argument is normally a tuple of types that you want to allow, but it's OK to pass a single type there of course (and, by making that one type an abstract base class, ABC, you can easily perform any kind of type-checking of your choice that way -- but, that's a different issue).

Here's some example code using this class:

x = TypedList((str, unicode), 'foo', 'bar')
x.append('zap')
print x
x.append(23)

the output is:

['foo', 'bar', 'zap']
Traceback (most recent call last):
  File "tl.py", line 35, in <module>
    x.append(23)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_abcoll.py", line 556, in append
    self.insert(len(self), value)
  File "tl.py", line 25, in insert
    self.check(v)
  File "tl.py", line 12, in check
    raise TypeError, v
TypeError: 23

Note in particular that we have not overridden append -- yet append is there and behaves just as expected.

The not-so-secret behind that bit of magic is revealed in the traceback: _abcoll.py (the implementation module for the abstract base classes in the collections module), at line 556, implements append by calling our insert -- which we have, of course, properly overridden.

This "template method design pattern" (absolutely precious for all kinds of OOP -- look for my talks on design patterns on youtube and you'll find out why;-), among other advantages, gives us the "choke point effect" I mentioned earlier: by adding some checks at a very few methods that you must implement, you gain the advantage that those checks apply to all the other relevant methods (and mutable sequences in Python have a lot of those;-).

It's no surprise that we end up with a very powerful and classic design pattern "behind the scenes", because the whole idea behind this implementation strategy comes right out of the immortal classic "Design Patterns" book (whose authors are often collectively referred to as the gang of four";-): prefer object composition over inheritance. Inheritance (from concrete classes) is a very rigid coupling mechanism, full of "gotchas" as soon as you're trying to use it to do anything even just slightly outside its rigorous limits; composition is extremely flexible and useful, and inheritance from appropriate abstract classes can complete the picture very nicely.

Scott Meyers' excellent "Effective C++", item 33, puts it even more strongly: make non-leaf classes abstract. Since by "non-leaf" he means "any class that's ever inherited from", an equivalent phrasing would be "never inherit from a concrete class".

Scott is writing in a C++ context, of course, but Paul Haahr gives exactly the same advice for Java, phrased as Don't subclass concrete classes -- and I generally second it for Python, though I do favor the gang-of-four's softer phrasing, prefer composition over (concrete class) inheritance (but I understand that both Scott and Paul are often writing for an audience which needs very direct and strongly phrased advice, almost phrased as "commandments" rather than advice, not softer phrased one that they might too easily ignore in the name of their convenience;-).

Answer 6

You can also use build-in array class. It works only with numeric types, but is probably best solution for such cases. It is optimized to minimize memory usage.

Example:

from array import array
array('c', 'hello world')     # char array
array('u', u'hello \u2641')   # unicode array
array('l', [1, 2, 3, 4, 5])   # long array
array('d', [1.0, 2.0, 3.14])  # double array

You can perform the same operations as with normal list:

chars = array('c')            
chars.extend('foo')

But when you try to insert other type then specified, an exception is raised:

>>> chars.extend([5,10])
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: array item must be char  

For all available types look here