How to use the same iterator twice, once for counting and once for iteration?

Question

It seems that an iterator is consumed when counting. How can I use the same iterator for counting and then iterate on it?

I\'m trying to count the lines in a file an

Answer 1

Iterators can generally not be iterated twice because there might be a cost to their iteration. In the case of str::lines, each iteration needs to find the next end of line, which means scanning through the string, which has some cost. You could argue that the iterator could save those positions for later reuse, but the cost of storing them would be even bigger.

Some Iterators are even more expensive to iterate, so you really don't want to do it twice.

Many iterators can be recreated easily (here calling str::lines a second time) or be cloned. Whichever way you recreate an iterator, the two iterators are generally completely independent, so iterating will mean you'll pay the price twice.

In your specific case, it is probably fine to just iterate the string twice as strings that fit in memory shouldn't be so long that merely counting lines would be a very expensive operation. If you believe this is the case, first benchmark it, second, write your own algorithm as Lines::count is probably not optimized as much as it could since the primary goal of Lines is to iterate lines.

Answer 2

Calling count consumes the iterator, because it actually iterates until it is done (i.e. next() returns None).

You can prevent consuming the iterator by using by_ref, but the iterator is still driven to its completion (by_ref actually just returns the mutable reference to the iterator, and Iterator is also implemented for the mutable reference: impl<'a, I> Iterator for &'a mut I).

This still can be useful if the iterator contains other state you want to reuse after it is done, but not in this case.

You could simply try forking the iterator (they often implement Clone if they don't have side effects), although in this case recreating it is just as good (most of the time creating an iterator is cheap; the real work is usually only done when you drive it by calling next directly or indirectly).

So no, (in this case) you can't reset it, and yes, you need to create a new one (or clone it before using it).

Answer 3

The other answers have already well-explained that you can either recreate your iterator or clone it.

If the act of iteration is overly expensive or it's impossible to do multiple times (such as reading from a network socket), an alternative solution is to create a collection of the iterator's values that will allow you to get the length and the values.

This does require storing every value from the iterator; there's no such thing as a free lunch!

use std::fs;

fn main() {
    let log_content = fs::read_to_string("/home/myuser/test.log").unwrap();
    let lines: Vec<_> = log_content.lines().collect();

    println!("{} lines", lines.len());
    for value in lines {
        println!("{}", value);
    }
}