exhausted iterators - what to do about them?

Question

(In Python 3.1) (Somewhat related to another question I asked, but this question is about iterators being exhausted.)

# trying to see the ratio of the max an         


        

Answer 1

The itertools.tee function can help here:

import itertools

f1, f2 = itertools.tee(filtered, 2)
ratio = max(f1) / min(f2)

Answer 2

The entity filtered is essentially an object with state. Of course, now it's obivous that running max or min on it will change that state. In order to stop tripping over that, I like to make it absolutely clear (to myself, really) that I am constructing something rather than just transforming something:

Adding an extra step can really help:

def filtered(container):
    return filter(lambda x : x is not None and x != 0, container)

ratio = max(filtered(c)) / min(filtered(c))

Whether you put filtered(...) inside some function (maybe it's not really needed for anything else) or define it as a module-level function is up to you, but in this case I'd suggest that if filtered (the iterator) was only needed in the function, leave it there until you need it elsewhere.

The other thing you can do is to construct a list from it, which will evaluate the iterator:

filtered_iter = filter(lambda x : x is not None and x != 0, container)
filtered = list(filtered_iter)

ratio = max(filtered) / min(filtered)

(Of course, you can just say filtered = list(filter(...)).)

Answer 3

you can convert an iterator to a tuple simply by calling tuple(iterator)

however I'd rewrite that filter as a list comprehension, which would look something like this

# original
filtered = filter(lambda x : x is not None and x != 0, c)

# list comp
filtered = [x for x in c if x is not None and x != 0]

Answer 4

Actually your code raises an exception that would prevent this problem! So I guess the problem was that you masked the exception?

>>> min([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: min() arg is an empty sequence
>>> min(x for x in ())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: min() arg is an empty sequence

Anyways, you can also write a new function to give you the min and max at the same time:

def minmax( seq ):
    " returns the `(min, max)` of sequence `seq`"
    it = iter(seq)
    try:
        min = max = next(it)
    except StopIteration:
        raise ValueError('arg is an empty sequence')
    for item in it:
        if item < min:
            min = item
        elif item > max:
            max = item
    return min, max