Why do Python yield statements form a closure?

Question

I have two functions that return a list of functions. The functions take in a number x and add i to it. i is an integer increasing from 0-

Answer 1

Adding to @sepp2k's answer you're seeing these two different behaviours because the lambda functions being created don't know from where they have to get i's value. At the time this function is created all it knows is that it has to either fetch i's value from either local scope, enclosed scope, global scope or builtins.

In this particular case it is a closure variable(enclosed scope). And its value is changing with each iteration.

---

Check out LEGB in Python.

---

Now to why second one works as expected but not the first one?

It's because each time you're yielding a lambda function the execution of the generator function stops at that moment and when you're invoking it and it will use the value of i at that moment. But in the first case we have already advanced i's value to 9 before we invoked any of the functions.

To prove it you can fetch current value of i from the __closure__'s cell contents:

>>> for func in test_with_yield():
        print "Current value of i is {}".format(func.__closure__[0].cell_contents)
        print func(9)
...
Current value of i is 0
Current value of i is 1
Current value of i is 2
Current value of i is 3
Current value of i is 4
Current value of i is 5
Current value of i is 6
...

But instead if you store the functions somewhere and call them later then you will see the same behaviour as the first time:

from itertools import islice

funcs = []
for func in islice(test_with_yield(), 4):
    print "Current value of i is {}".format(func.__closure__[0].cell_contents)
    funcs.append(func)

print '-' * 20

for func in funcs:
    print "Now value of i is {}".format(func.__closure__[0].cell_contents)

Output:

Current value of i is 0
Current value of i is 1
Current value of i is 2
Current value of i is 3
--------------------
Now value of i is 3
Now value of i is 3
Now value of i is 3
Now value of i is 3

Example used by Patrick Haugh in comments also shows the same thing: sum(t(1) for t in list(test_with_yield()))

---

Correct way:

Assign i as a default value to lambda, default values are calculated when function is created and they won't change(unless it's a mutable object). i is now a local variable to the lambda functions.

>>> def test_without_closure():
        return [lambda x, i=i: x+i for i in range(10)]
...
>>> sum(t(1) for t in test_without_closure())
55