问题
I'm trying to create the Cartesian product of a list of lists. When I try to convert the result to a list it will give me a memory error. If I run it without converting it to a list it runs fine.
lists = [['a','b','c' ],['a','b','c' ],['a','b','c' ],['a','b','c' ],['a','b','c' ],['a','b','c' ],['a','b','c' ],['a','b','c' ],['a','b','c' ]]
my_product = list(itertools.product(*lists))
I even tried to filter through some of the result using itertools.dropwhile to make it smaller before converting it into a list and I get the same result.
filtered = itertools.dropwhile(lambda x: x[1]!=x[2] and x[3]!=x[4] and x[3]!=x[5] and x[4]!=x[5], my_product)
回答1:
You are creating 19683 new tuples of 9 elements. Your computer does not have enough memory available for all those tuples at once.
If you do not use list(), then only a generator object is created, one that will produce the 19683 one-by-one as you iterate over it.
You should filter the output of itertools.product() without converting it to a list:
my_product = itertools.product(*lists)
filtered = itertools.dropwhile(lambda x: x[1]!=x[2] and x[3]!=x[4] and x[3]!=x[5] and x[4]!=x[5], my_product)
You could turn filtered into a list, but if all you do is loop over that list and process items one by one, you should not do that. Only turn it into a list if you need random access to the elements.
来源：https://stackoverflow.com/questions/18390336/cartesian-product-memory-error-converting-itertools-product-to-list