How can I read large text files in Python, line by line, without loading it into memory?

前端 未结 15 1269
臣服心动
臣服心动 2020-11-22 03:32

I need to read a large file, line by line. Lets say that file has more than 5GB and I need to read each line, but obviously I do not want to use readlines() bec

15条回答
  •  抹茶落季
    2020-11-22 04:21

    The best solution I found regarding this, and I tried it on 330 MB file.

    lineno = 500
    line_length = 8
    with open('catfour.txt', 'r') as file:
        file.seek(lineno * (line_length + 2))
        print(file.readline(), end='')
    

    Where line_length is the number of characters in a single line. For example "abcd" has line length 4.

    I have added 2 in line length to skip the '\n' character and move to the next character.

提交回复
热议问题