Length of a finite generator

Question

I have these two implementations to compute the length of a finite generator, while keeping the data for further processing:

def count_generator1(generator):         


        

Answer 1

If you don't need the length of the iterator prior to processing the data, you could use a helper method with a future to add in counting into the processing of your iterator/stream:

import asyncio
def ilen(iter):
    """
    Get future with length of iterator
    The future will hold the length once the iteartor is exhausted
    @returns: 
    """
    def ilen_inner(iter, future):
        cnt = 0
        for row in iter:
            cnt += 1
            yield row
        future.set_result(cnt)
    cnt_future = asyncio.Future()
    return ilen_inner(iter, cnt_future), cnt_future

Usage would be:

data = db_connection.execute(query)
data, cnt = ilen(data)
solve_world_hunger(data)
print(f"Processed {cnt.result()} items")