CSVWriter not saving data to file the moment I write it

Question

Python newbie getting a bit frustrated with the csv module. At this rate, it would have been easier if I wrote the file parser myself, but I want to do things the Pythonic w

Answer 1

Use

with open('myfile.csv','wb') as myfile:
    wrtr = csv.writer(myfile, delimiter=',', quotechar='"')
    for row in rows:
        wrtr.writerow([row.field1,row.field2,row.field3])
        myfile.flush() # whenever you want

or

myfile = open('myfile.csv','wb')
wrtr = csv.writer(myfile, delimiter=',', quotechar='"')
for row in rows:
    wrtr.writerow([row.field1,row.field2,row.field3])
    myfile.flush() # whenever you want, and/or
myfile.close() # when you're done.

The nice thing about the first approach is that your file will also be automatically properly closed in case of an Exception.

If you want your file object to be anonymous, then it will only be closed when the program exits. When or whether it is flushed depends on the OS - so it might be never until exit.

Answer 2

Wow, I was in the wrong place, skip this answer!

I've had the same problem. Writing 1000 records in 1 WriteRecords call, but it would leave the last record(s?) written incomplete.

Even calling like fileStream.Flush(); does not work.

When you do the following, it will flush it correctly:

csvWriter.Flush();
csvWriter.Context.Writer.Flush();

Seen on: https://github.com/JoshClose/CsvHelper/issues/967#issuecomment-482503533

Answer 3

The flush() and close() methods of the file object. Or use with.