Iterating on a file doesn't work the second time

Question

I have a problem with iterating on a file. Here\'s what I type on the interpreter and the result:

>>> f = open(\'baby1990.html\', \'rU\')
>>&g         


        

Answer 1

Yes, that is normal behavior. You basically read to the end of the file the first time (you can sort of picture it as reading a tape), so you can't read any more from it unless you reset it, by either using f.seek(0) to reposition to the start of the file, or to close it and then open it again which will start from the beginning of the file.

If you prefer you can use the with syntax instead which will automatically close the file for you.

e.g.,

with open('baby1990.html', 'rU') as f:
  for line in f:
     print line

once this block is finished executing, the file is automatically closed for you, so you could execute this block repeatedly without explicitly closing the file yourself and read the file this way over again.

Answer 2

Of course. That is normal and sane behaviour. Instead of closing and re-opening, you could rewind the file.

Answer 3

The file object is a buffer. When you read from the buffer, that portion that you read is consumed (the read position is shifted forward). When you read through the entire file, the read position is at the end of the file (EOF), so it returns nothing because there is nothing left to read.

If you have to reset the read position on a file object for some reason, you can do:

f.seek(0)

Answer 4

As the file object reads the file, it uses a pointer to keep track of where it is. If you read part of the file, then go back to it later it will pick up where you left off. If you read the whole file, and go back to the same file object, it will be like reading an empty file because the pointer is at the end of the file and there is nothing left to read. You can use file.tell() to see where in the file the pointer is and file.seek to set the pointer. For example:

>>> file = open('myfile.txt')
>>> file.tell()
0
>>> file.readline()
'one\n'
>>> file.tell()
4L
>>> file.readline()
'2\n'
>>> file.tell()
6L
>>> file.seek(4)
>>> file.readline()
'2\n'

Also, you should know that file.readlines() reads the whole file and stores it as a list. That's useful to know because you can replace:

for line in file.readlines():
    #do stuff
file.seek(0)
for line in file.readlines():
    #do more stuff

with:

lines = file.readlines()
for each_line in lines:
    #do stuff
for each_line in lines:
    #do more stuff

You can also iterate over a file, one line at a time, without holding the whole file in memory (this can be very useful for very large files) by doing:

for line in file:
    #do stuff