efficiency of Python's itertools.product()

问题

So I'm looking at different ways to compute the Cartesian product of n arrays, and I came across the rather elegant solution (here on SO) of using the following code:

import itertools
    for array in itertools.product(*arrays):
        print array

Looking at the python doc page (I'm using 2.7, btw) for itertools.product(), it says the code is equivalent to the following:

def product(*args, **kwds):
    # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy
    # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111
    pools = map(tuple, args) * kwds.get('repeat', 1)
    result = [[]]
    for pool in pools:
        result = [x+[y] for x in result for y in pool]
    for prod in result:
        yield tuple(prod)

(It does note the following: This function is equivalent to the following code, except that the actual implementation does not build up intermediate results in memory:)

I'm not a CS person - so I'm pretty bad at estimating the efficiency of this algorithm. My first guess would be O(n^2) (due to the nested for loop).

Am I wrong?

回答1:

You are absolutely right. That is, in the special case of two arrays input, both of the size n. In the general case of k arrays of the sizes n[i] for i in 1..k it will be O(Product of all n[i]).

Why is this the case and why is there no way to optimize this any further?

Well, in this case the size of the output is directly this "Product of all n[i]" which lies in the nature of the function we are discussing. Python makes this even more obvious by implementing it as a generator. So for each element, this generator yields one element, in the end it will be as many yielded elements as the said product.

Of course, if something so obviously does anything x times, its efficiency cannot be better than O(x). It could be worse if the effort for each element was also depending on the input size. So, to be precise, the effort for each element here is depending on the number of arrays we put in, so the true effort would be

O(k × Product of all n[i])

来源：https://stackoverflow.com/questions/31960583/efficiency-of-pythons-itertools-product

标签

python

algorithm

performance

cartesian-product

set-theory