How to compose iterators?

Question

I have a network of nodes passing structured data in between. For my subproblem, we have this branch - linear sequence of nodes:

nodes = [source, n1, n2, n3,         


        

Answer 1

If all you want is to compose arbitrary numbers of functions (or callables), use the compose_mult recipe from the functional module documentation.

A solution which uses that:

from functional import compose, foldr, partial
from itertools  import imap
compose_mult = partial(reduce, compose) 
chain_nodes = lambda nodes: imap(compose_mult(nodes[1:]), nodes[0])
chain_gen_nodes = lambda nodes: imap(compose_mult((g.send for g in nodes[1:])), nodes[0])


# equivalent not as a one-liner
#def chain_nodes(nodes):
#    source = nodes[0]
#    composed_nodes = compose_mult(nodes[1:])
#    return (composed_nodes(x) for x in source)

If the nodes are generators that accept input (via send), then use chain_gen_nodes, which extracts their send function.

Note, however, that one is not allowed to send to a just-started generator (because it has to be at the point of a yield to receive the value). This is something you are going to have to handle yourself, such as by having your generators yield a dummy value on their first iteration, and advancing them at some point before sending them to chain_nodes. Or you could just keep your nodes as ordinary callables.

If you do need to advance the iterators one step: next(izip(*nodes[1:]))