namedtuple and default values for optional keyword arguments

问题

I\'m trying to convert a longish hollow \"data\" class into a named tuple. My class currently looks like this:

class Node(object):
    def __init__(self, val, left=None, right=None):
        self.val = val
        self.left = left
        self.right = right

After conversion to namedtuple it looks like:

from collections import namedtuple
Node = namedtuple(\'Node\', \'val left right\')

But there is a problem here. My original class allowed me to pass in just a value and took care of the default by using default values for the named/keyword arguments. Something like:

class BinaryTree(object):
    def __init__(self, val):
        self.root = Node(val)

But this doesn\'t work in the case of my refactored named tuple since it expects me to pass all the fields. I can of course replace the occurrences of Node(val) to Node(val, None, None) but it isn\'t to my liking.

So does there exist a good trick which can make my re-write successful without adding a lot of code complexity (metaprogramming) or should I just swallow the pill and go ahead with the \"search and replace\"? :)

回答1:

Python 3.7

Use the defaults parameter.

>>> from collections import namedtuple
>>> fields = ('val', 'left', 'right')
>>> Node = namedtuple('Node', fields, defaults=(None,) * len(fields))
>>> Node()
Node(val=None, left=None, right=None)

Before Python 3.7

Set Node.__new__.__defaults__ to the default values.

>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.__defaults__ = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)

Before Python 2.6

Set Node.__new__.func_defaults to the default values.

>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.func_defaults = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)

Order

In all versions of Python, if you set fewer default values than exist in the namedtuple, the defaults are applied to the rightmost parameters. This allows you to keep some arguments as required arguments.

>>> Node.__new__.__defaults__ = (1,2)
>>> Node()
Traceback (most recent call last):
  ...
TypeError: __new__() missing 1 required positional argument: 'val'
>>> Node(3)
Node(val=3, left=1, right=2)

Wrapper for Python 2.6 to 3.6

Here's a wrapper for you, which even lets you (optionally) set the default values to something other than None. This does not support required arguments.

import collections
def namedtuple_with_defaults(typename, field_names, default_values=()):
    T = collections.namedtuple(typename, field_names)
    T.__new__.__defaults__ = (None,) * len(T._fields)
    if isinstance(default_values, collections.Mapping):
        prototype = T(**default_values)
    else:
        prototype = T(*default_values)
    T.__new__.__defaults__ = tuple(prototype)
    return T

Example:

>>> Node = namedtuple_with_defaults('Node', 'val left right')
>>> Node()
Node(val=None, left=None, right=None)
>>> Node = namedtuple_with_defaults('Node', 'val left right', [1, 2, 3])
>>> Node()
Node(val=1, left=2, right=3)
>>> Node = namedtuple_with_defaults('Node', 'val left right', {'right':7})
>>> Node()
Node(val=None, left=None, right=7)
>>> Node(4)
Node(val=4, left=None, right=7)

回答2:

I subclassed namedtuple and overrode the __new__ method:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, value, left=None, right=None):
        return super(Node, cls).__new__(cls, value, left, right)

This preserves an intuitive type hierarchy, which the creation of a factory function disguised as a class does not.

回答3:

Wrap it in a function.

NodeT = namedtuple('Node', 'val left right')

def Node(val, left=None, right=None):
  return NodeT(val, left, right)

回答4:

With typing.NamedTuple in Python 3.6.1+ you can provide both a default value and a type annotation to a NamedTuple field. Use typing.Any if you only need the former:

from typing import Any, NamedTuple


class Node(NamedTuple):
    val: Any
    left: 'Node' = None
    right: 'Node' = None

Usage:

>>> Node(1)
Node(val=1, left=None, right=None)
>>> n = Node(1)
>>> Node(2, left=n)
Node(val=2, left=Node(val=1, left=None, right=None), right=None)

Also, in case you need both default values and optional mutability, Python 3.7 is going to have data classes (PEP 557) that can in some (many?) cases replace namedtuples.

---

Sidenote: one quirk of the current specification of annotations (expressions after : for parameters and variables and after -> for functions) in Python is that they are evaluated at definition time^*. So, since "class names become defined once the entire body of the class has been executed", the annotations for 'Node' in the class fields above must be strings to avoid NameError.

This kind of type hints is called "forward reference" ([1], [2]), and with PEP 563 Python 3.7+ is going to have a __future__ import (to be enabled by default in 4.0) that will allow to use forward references without quotes, postponing their evaluation.

^{* AFAICT only local variable annotations are not evaluated at runtime. (source: PEP 526)}

回答5:

This is an example straight from the docs:

Default values can be implemented by using _replace() to customize a prototype instance:

>>> Account = namedtuple('Account', 'owner balance transaction_count')
>>> default_account = Account('<owner name>', 0.0, 0)
>>> johns_account = default_account._replace(owner='John')
>>> janes_account = default_account._replace(owner='Jane')

So, the OP's example would be:

from collections import namedtuple
Node = namedtuple('Node', 'val left right')
default_node = Node(None, None, None)
example = default_node._replace(val="whut")

However, I like some of the other answers given here better. I just wanted to add this for completeness.

回答6:

I'm not sure if there's an easy way with just the built-in namedtuple. There's a nice module called recordtype that has this functionality:

>>> from recordtype import recordtype
>>> Node = recordtype('Node', [('val', None), ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)

回答7:

Here is a more compact version inspired by justinfay's answer:

from collections import namedtuple
from functools import partial

Node = namedtuple('Node', ('val left right'))
Node.__new__ = partial(Node.__new__, left=None, right=None)

回答8:

In python3.7+ there's a brand new defaults= keyword argument.

defaults can be None or an iterable of default values. Since fields with a default value must come after any fields without a default, the defaults are applied to the rightmost parameters. For example, if the fieldnames are ['x', 'y', 'z'] and the defaults are (1, 2), then x will be a required argument, y will default to 1, and z will default to 2.

Example usage:

$ ./python
Python 3.7.0b1+ (heads/3.7:4d65430, Feb  1 2018, 09:28:35) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import namedtuple
>>> nt = namedtuple('nt', ('a', 'b', 'c'), defaults=(1, 2))
>>> nt(0)
nt(a=0, b=1, c=2)
>>> nt(0, 3)  
nt(a=0, b=3, c=2)
>>> nt(0, c=3)
nt(a=0, b=1, c=3)

回答9:

Short, simple, and doesn't lead people to use isinstance improperly:

class Node(namedtuple('Node', ('val', 'left', 'right'))):
    @classmethod
    def make(cls, val, left=None, right=None):
        return cls(val, left, right)

# Example
x = Node.make(3)
x._replace(right=Node.make(4))

回答10:

A slightly extended example to initialize all missing arguments with None:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, *args, **kwargs):
        # initialize missing kwargs with None
        all_kwargs = {key: kwargs.get(key) for key in cls._fields}
        return super(Node, cls).__new__(cls, *args, **all_kwargs)

回答11:

Python 3.7: introduction of defaults param in namedtuple definition.

Example as shown in the documentation:

>>> Account = namedtuple('Account', ['type', 'balance'], defaults=[0])
>>> Account._fields_defaults
{'balance': 0}
>>> Account('premium')
Account(type='premium', balance=0)

Read more here.

回答12:

You can also use this:

import inspect

def namedtuple_with_defaults(type, default_value=None, **kwargs):
    args_list = inspect.getargspec(type.__new__).args[1:]
    params = dict([(x, default_value) for x in args_list])
    params.update(kwargs)

    return type(**params)

This basically gives you the possibility to construct any named tuple with a default value and override just the parameters you need, for example:

import collections

Point = collections.namedtuple("Point", ["x", "y"])
namedtuple_with_defaults(Point)
>>> Point(x=None, y=None)

namedtuple_with_defaults(Point, x=1)
>>> Point(x=1, y=None)

回答13:

Combining approaches of @Denis and @Mark:

from collections import namedtuple
import inspect

class Node(namedtuple('Node', 'left right val')):
    __slots__ = ()
    def __new__(cls, *args, **kwargs):
        args_list = inspect.getargspec(super(Node, cls).__new__).args[len(args)+1:]
        params = {key: kwargs.get(key) for key in args_list + kwargs.keys()}
        return super(Node, cls).__new__(cls, *args, **params) 

That should support creating the tuple with positional arguments and also with mixed cases. Test cases:

>>> print Node()
Node(left=None, right=None, val=None)

>>> print Node(1,2,3)
Node(left=1, right=2, val=3)

>>> print Node(1, right=2)
Node(left=1, right=2, val=None)

>>> print Node(1, right=2, val=100)
Node(left=1, right=2, val=100)

>>> print Node(left=1, right=2, val=100)
Node(left=1, right=2, val=100)

>>> print Node(left=1, right=2)
Node(left=1, right=2, val=None)

but also support TypeError:

>>> Node(1, left=2)
TypeError: __new__() got multiple values for keyword argument 'left'

回答14:

I find this version easier to read:

from collections import namedtuple

def my_tuple(**kwargs):
    defaults = {
        'a': 2.0,
        'b': True,
        'c': "hello",
    }
    default_tuple = namedtuple('MY_TUPLE', ' '.join(defaults.keys()))(*defaults.values())
    return default_tuple._replace(**kwargs)

This is not as efficient as it requires creation of the object twice but you could change that by defining the default duple inside the module and just having the function do the replace line.

回答15:

Since you are using namedtuple as a data class, you should be aware that python 3.7 will introduce a @dataclass decorator for this very purpose -- and of course it has default values.

An example from the docs:

@dataclass
class C:
    a: int       # 'a' has no default value
    b: int = 0   # assign a default value for 'b'

Much cleaner, readable and usable than hacking namedtuple. It is not hard to predict that usage of namedtuples will drop with the adoption of 3.7.

回答16:

Inspired by this answer to a different question, here is my proposed solution based on a metaclass and using super (to handle future subcalssing correctly). It is quite similar to justinfay's answer.

from collections import namedtuple

NodeTuple = namedtuple("NodeTuple", ("val", "left", "right"))

class NodeMeta(type):
    def __call__(cls, val, left=None, right=None):
        return super(NodeMeta, cls).__call__(val, left, right)

class Node(NodeTuple, metaclass=NodeMeta):
    __slots__ = ()

Then:

>>> Node(1, Node(2, Node(4)),(Node(3, None, Node(5))))
Node(val=1, left=Node(val=2, left=Node(val=4, left=None, right=None), right=None), right=Node(val=3, left=None, right=Node(val=5, left=None, right=None)))

回答17:

The answer by jterrace to use recordtype is great, but the author of the library recommends to use his namedlist project, which provides both mutable (namedlist) and immutable (namedtuple) implementations.

from namedlist import namedtuple
>>> Node = namedtuple('Node', ['val', ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)

回答18:

Here's a short, simple generic answer with a nice syntax for a named tuple with default arguments:

import collections

def dnamedtuple(typename, field_names, **defaults):
    fields = sorted(field_names.split(), key=lambda x: x in defaults)
    T = collections.namedtuple(typename, ' '.join(fields))
    T.__new__.__defaults__ = tuple(defaults[field] for field in fields[-len(defaults):])
    return T

Usage:

Test = dnamedtuple('Test', 'one two three', two=2)
Test(1, 3)  # Test(one=1, three=3, two=2)

Minified:

def dnamedtuple(tp, fs, **df):
    fs = sorted(fs.split(), key=df.__contains__)
    T = collections.namedtuple(tp, ' '.join(fs))
    T.__new__.__defaults__ = tuple(df[i] for i in fs[-len(df):])
    return T

回答19:

Using the NamedTuple class from my Advanced Enum (aenum) library, and using the class syntax, this is quite simple:

from aenum import NamedTuple

class Node(NamedTuple):
    val = 0
    left = 1, 'previous Node', None
    right = 2, 'next Node', None

The one potential drawback is the requirement for a __doc__ string for any attribute with a default value (it's optional for simple attributes). In use it looks like:

>>> Node()
Traceback (most recent call last):
  ...
TypeError: values not provided for field(s): val

>>> Node(3)
Node(val=3, left=None, right=None)

The advantages this has over justinfay's answer:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, value, left=None, right=None):
        return super(Node, cls).__new__(cls, value, left, right)

is simplicity, as well as being metaclass based instead of exec based.

回答20:

Another solution:

import collections


def defaultargs(func, defaults):
    def wrapper(*args, **kwargs):
        for key, value in (x for x in defaults[len(args):] if len(x) == 2):
            kwargs.setdefault(key, value)
        return func(*args, **kwargs)
    return wrapper


def namedtuple(name, fields):
    NamedTuple = collections.namedtuple(name, [x[0] for x in fields])
    NamedTuple.__new__ = defaultargs(NamedTuple.__new__, [(NamedTuple,)] + fields)
    return NamedTuple

Usage:

>>> Node = namedtuple('Node', [
...     ('val',),
...     ('left', None),
...     ('right', None),
... ])
__main__.Node

>>> Node(1)
Node(val=1, left=None, right=None)

>>> Node(1, 2, right=3)
Node(val=1, left=2, right=3)

回答21:

Here's a less flexible, but more concise version of Mark Lodato's wrapper: It takes the fields and defaults as a dictionary.

import collections
def namedtuple_with_defaults(typename, fields_dict):
    T = collections.namedtuple(typename, ' '.join(fields_dict.keys()))
    T.__new__.__defaults__ = tuple(fields_dict.values())
    return T

Example:

In[1]: fields = {'val': 1, 'left': 2, 'right':3}

In[2]: Node = namedtuple_with_defaults('Node', fields)

In[3]: Node()
Out[3]: Node(val=1, left=2, right=3)

In[4]: Node(4,5,6)
Out[4]: Node(val=4, left=5, right=6)

In[5]: Node(val=10)
Out[5]: Node(val=10, left=2, right=3)

来源：https://stackoverflow.com/questions/11351032/namedtuple-and-default-values-for-optional-keyword-arguments