Efficient creation of numpy arrays from list comprehension and in general

Question

In my current work i use Numpy and list comprehensions a lot and in the interest of best possible performance i have the following questions:

What actually happens b

Answer 1

I believe than answer you are looking for is using generator expressions with numpy.fromiter.

numpy.fromiter((<some_func>(x) for x in <something>),<dtype>,<size of something>)

Generator expressions are lazy - they evaluate the expression when you iterate through them.

Using list comprehensions makes the list, then feeds it into numpy, while generator expressions will yield one at a time.

Python evaluates things inside -> out, like most languages (if not all), so using [<something> for <something_else> in <something_different>] would make the list, then iterate over it.

Answer 2

You could create your own list and experiment with it to shed some light on the situation...

>>> class my_list(list):
...     def __init__(self, arg):
...         print 'spam'
...         super(my_list, self).__init__(arg)
...   def __len__(self):
...       print 'eggs'
...       return super(my_list, self).__len__()
... 
>>> x = my_list([0,1,2,3])
spam
>>> len(x)
eggs
4
>>> import numpy as np
>>> np.array(x)
eggs
eggs
eggs
eggs
array([0, 1, 2, 3])
>>> np.fromiter(x, int)
array([0, 1, 2, 3])
>>> np.array(my_list([0,1,2,3]))
spam
eggs
eggs
eggs
eggs
array([0, 1, 2, 3])