ipython %timeit “local variable 'a' referenced before assignment”

Question

I am trying to the run the following code but I get a local variable \'a\' referenced before assignment.

a = [x for x in range(10)]
b = [x for          


        

Answer 1

What do you expect it to do?

Outside of the timeit it does:

In [188]: a += b
In [189]: a
Out[189]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

I tried initializing x, and got a near infinite loop, eventually ending with a memory error

In [192]: %%timeit x=a
     ...: x += b

In [194]: len(a)
Out[194]: 529076630

In other words each timeit loop concatenated another list of b values to x (and by extension a), resulting are very long loop. I suspect an individual x+=b was fast, resulting in timeit choosing to loop many times.

Lets create an a fresh each loop:

In [196]: %%timeit
     ...: a = [x for x in range(10)]
     ...: a += b
     ...: 
1.91 µs ± 4.82 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

This produces the memory error as well:

In [197]: %%timeit a = [x for x in range(10)]
     ...: a += b

If I control the number of loops:

In [202]: %%timeit -n 100 a = [x for x in range(10)]
     ...: a += b
     ...: 
     ...: 
208 ns ± 11 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)

With ns times I can see why the default loops is so large.

I haven't tried timing a plain a+=... before (not even with numpy arrays), but evidently it expects some sort of local initialization for that a, either within the loop or in the initialization block. But it is important to keep in mind that the timed actions may be performed many times (the -r and -n parameters or the default values). So any in-place action might result bit changes to the global values. In this case timeit might be trying to protect us from that kind of unexpected growth, by expecting some sort of 'local' variable.

---

Lets try the a+b, but with an assignment:

In [215]: c=np.zeros(10)
In [216]: a
Out[216]: array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
In [217]: b
Out[217]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [218]: %timeit c = a+b
5.33 µs ± 105 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [219]: c
Out[219]: array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])

Notice that the global c has not changed. The assignment is to a temporary local c - even though a global of the same name is available.

As a general rule, calculations performed inside the timing loop should not leak outside the loop. You have to something explicit as I did in the memory error loop, or here

In [222]: %%timeit x = c
     ...: x += b
     ...: 
9.04 µs ± 238 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [223]: c
Out[223]: 
array([       0.,   811111.,  1622222.,  2433333.,  3244444.,  4055555.,
        4866666.,  5677777.,  6488888.,  7299999.])

or here:

In [224]: c=np.zeros(10)
In [225]: %%timeit x = c
     ...: x[:] = a+b

7.84 µs ± 199 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [226]: c
Out[226]: array([  1.,   3.,   5.,   7.,   9.,  11.,  13.,  15.,  17.,  19.])

Both use an in-place assignment to a local variable which has been linked to a mutable global.