How can I read large text files in Python, line by line, without loading it into memory?

Question

I need to read a large file, line by line. Lets say that file has more than 5GB and I need to read each line, but obviously I do not want to use readlines() bec

Answer 1

Here's what you do if you dont have newlines in the file:

with open('large_text.txt') as f:
  while True:
    c = f.read(1024)
    if not c:
      break
    print(c)

Answer 2

The best solution I found regarding this, and I tried it on 330 MB file.

lineno = 500
line_length = 8
with open('catfour.txt', 'r') as file:
    file.seek(lineno * (line_length + 2))
    print(file.readline(), end='')

Where line_length is the number of characters in a single line. For example "abcd" has line length 4.

I have added 2 in line length to skip the '\n' character and move to the next character.

Answer 3

f=open('filename','r').read()
f1=f.split('\n')
for i in range (len(f1)):
    do_something_with(f1[i])

hope this helps.