Python: Map a function over recursive iterables

Question

I have an arbitrarily nested iterable like so:

numbers = (1, 2, (3, (4, 5)), 7)

and I\'d like to map a function over it without changing the st

Answer 1

Everyone before has mentioned the number of things one might possibly need for any flavour of a flatten function, but there was something that I've been playing with as an exercise in learning the language (so Python noob alert) that I didn't see quite put together here. Basically I wanted for my flatten to be able to handle any Iterables, of any length and nesting in the most efficient (time and space) way possible. This lead me to the generator pattern, and the first requirement I posed for my function was nothing to be created before its time.

My other requirement was the absence of any explicit looping (for/while) because why not: at least since the helpful addition of yield from in Python 3.3 I was pretty sure it was possible. It would have to be recursive of course, but getting it to give a proper, "flat" generator proved trickier than I thought. So here's my 2p, illustrating the wonderful chain and, I suspect, the kind of situation (a bit more abstracted of course) it was made for:

from itertools import chain
from collections import Iterable

def flatten(items):
    if isinstance(items,Iterable):
        yield from chain(*map(flatten,items))    
    else:
        yield items

items = [0xf, [11, 22, [23, (33,(4, 5))], 66, [], [77]], [8,8], 99, {42}]
print(list(flatten(items)))

Unfortunately for my for-free ambitious project (and ego), according to some pretty rough benchmarking this is ~30% slower than the version using for:

def flatten(items):
    for item in items:
        if isinstance(item,Iterable):
            yield from flatten(item)
        else:
            yield item

a variant of which was already given by Uriel. I hope it is however a good illustration of the flexibility and power of Python used in a quasi-functional way, especially for others new to the language.

Edit: to avoid splitting up strings in individual list items, one can append and not isinstance(item,(str,bytes)) to the conditional. And other various bells and whistles that would detract from the point.

Answer 2

I extended the notion of a recursive map to work on the standard python collections: list, dict, set, tuple:

def recursiveMap(something, func):
  if isinstance(something, dict):
    accumulator = {}
    for key, value in something.items():
      accumulator[key] = recursiveMap(value, func)
    return accumulator
  elif isinstance(something, (list, tuple, set)):
    accumulator = []
    for item in something:
      accumulator.append(recursiveMap(item, func))
    return type(something)(accumulator)
  else:
    return func(something)

This passes the following tests, which I'll include mostly as examples of usage:

from hypothesis                 import given
from hypothesis.strategies      import dictionaries, text
from server.utils               import recursiveMap


def test_recursiveMap_example_str():
  assert recursiveMap({'a': 1}, str) == {'a': '1'}
  assert recursiveMap({1: 1}, str) == {1: '1'}
  assert recursiveMap({'a': {'a1': 12}, 'b': 2}, str) == {'a': {'a1': '12'}, 'b': '2'}
  assert recursiveMap([1, 2, [31, 32], 4], str) == ['1', '2', ['31', '32'], '4']
  assert recursiveMap((1, 2, (31, 32), 4), str) ==  ('1', '2', ('31', '32'), '4')
  assert recursiveMap([1, 2, (31, 32), 4], str) ==  ['1', '2', ('31', '32'), '4']


@given(dictionaries(text(), text()))
def test_recursiveMap_noop(dictionary):
  assert recursiveMap(dictionary, lambda x: x) == dictionary

Answer 3

If you want to extend your result to dict, set and others, you can use Uriel's answer:

from collections import Collection, Mapping

def recursive_map(data, func):
    apply = lambda x: recursive_map(x, func)
    if isinstance(data, Mapping):
        return type(data)({k: apply(v) for k, v in data.items()})
    elif isinstance(data, Collection):
        return type(data)(apply(v) for v in data)
    else:
        return func(data)

Tests input:

recursive_map({0: [1, {2, 2, 3}]}, str)

Yields:

{0: ['1', '{2, 3}']}

Answer 4

def recursive_map(f, it):
    return (recursive_map(f, x) if isinstance(x, tuple) else f(x) for x in it)

Answer 5

We scan every element in the sequence, and proceeds into deeper recursion if the current item is a sub-sequence, or yields it's mapping if we reached a non-sequence data type (could be int, str, or any complex classes).

We use collections.Sequence to generalize the idea for every sequence, and not only tuples or lists, and type(item) upon yield to ensure that the sub-sequences we get back remains of the same type they were.

from collections import Sequence

def recursive_map (seq, func):
    for item in seq:
        if isinstance(item, Sequence):
            yield type(item)(recursive_map(item, func))
        else:
            yield func(item)

Demo:

>>> numbers = (1, 2, (3, (4, 5)), 7)
>>> mapped = recursive_map(numbers, str)
>>> tuple(mapped)
('1', '2', ('3', ('4', '5')), '7')

Or a more complex example:

>>> complex_list = (1, 2, [3, (complex('4+2j'), 5)], map(str, (range(7, 10))))
>>> tuple(recursive_map(complex_list, lambda x: x.__class__.__name__))
('int', 'int', ['int', ('complex', 'int')], 'map')