Python: Force heapq.merge to interpert strings as integers during comparison

Question

I\'m attempting to merge a group of pre-sorted files where every line in each file is an integer:

for line in heapq.merge(*files):

The sort com

Answer 1

Try this:

for line in heapq.merge(*(map(int, file) for file in files)):

That doesn't interpret the strings as integers during comparison, but actually on-the-fly changes them to integers. The outcome is therefore integers, not strings. Can of course then be converted back to strings if desirable:

for line in map(str, heapq.merge(*(map(int, file) for file in files))):

For others / future reference: This is for Python 3, where map returns an iterator. In Python 2, map would need to be replaced by itertools.imap in order to not read everything into memory at startup.

Answer 2

Try reading the files and converting each line into an integer. This assumes that all data fit into memory.

def read_as_int_list(file_name):
    with open(file_name) as fobj:
        return [int(line) for line in fobj]

This should be more memory efficient:

def read_as_ints(file_name):
    with open(file_name) as fobj:
        for line in fobj:
            yield int(line)

Usage:

files = (read_as_ints(name) for name in list_of_file_names)
for line in heapq.merge(*files):
    print(line)