Idiom for flattening a shallow nested list: how does it work?

Question

I found this bit of code in a module I am working on:

l = opaque_function()
thingys = [x for y in l for x in y]

I can\'t read this. By expe

Answer 1

This used to really confuse me. You should read it like a nested loop:

new_list = []
for y in l:
    for x in y:
        new_list.append(x)

becomes

for y in l for x in y [do] new_list.append(x)

becomes

[x for y in l for x in y]

Answer 2

You should read this as:

for y in l:
    for x in y:
        yield x

That's the generator version, but all comprehensions have the same basic syntax: while the x is put up front, the rest of the expression is still read left-to-right. I was confused by this at first too, expecting it to be the other way around, but it makes sense once you add filtering expressions:

>>> l = [[1,2,3,4,5], [1,"foo","bar"], [2,3]]
>>> [x for y in l
...    if len(y) < 4
...    for x in y
...    if isinstance(x, int)]
[1, 2, 3]

Now imagine having to write this entire thing backwards:

[x if isinstance(x, int)
   for x in y
   if len(y) < 4
   for y in l]

That would be confusing even to veteran Prolog programmers, not to mention the people maintaining Python parsers :)

The current syntax also matches that in Haskell, which inspired list comprehensions in the first place.

Answer 3

From the list displays documentation:

When a list comprehension is supplied, it consists of a single expression followed by at least one for clause and zero or more for or if clauses. In this case, the elements of the new list are those that would be produced by considering each of the for or if clauses a block, nesting from left to right, and evaluating the expression to produce a list element each time the innermost block is reached.

Thus, your expression can be rewritten as:

thingys = []
for y in l:
    for x in y:
        thingys.append(x)

Answer 4

lis=[x for y in l for x in y] is Equivalent to:


lis=[]
for y in l:
   for x in y:
      lis.append(x)