Why this list comprehension is faster than equivalent generator expression?

Question

I\'m using Python 3.3.1 64-bit on Windows and this code snippet:

len ([None for n in range (1, 1000000) if n%3 == 1])

executes in 136ms, c

Answer 1

Borrowed from this answer, there are two things to consider:

1. A Python list is index-able and fetching its length only takes O(1) times. This means that the speed of calling len() on a list does not depend on its size. However, if you call len() on a generator, you're consuming all the items it generates and thus, the time complexity is O(n).

2. See the linked answer above. A list comprehension is a tight C loop, whereas a generator has to store a reference to the iterator inside and call next(iter) for every item it generates. This creates another layer of overhead for generators. At a small scale, the difference in performance between list comprehension and generators can be safely ignored, but at a larger scale, you have to consider this.