How to deal with a very large text file?

Question

I\'m currently writing something that needs to handle very large text files (a few GiB at least). What\'s needed here (and this is fixed) is:

• CSV-based, following

Answer 1

If you had fixed width lines then using a RandomAccessFile might solve a lot of your problems. I realise that your lines are probably not fixed width, but you could artificially impose this by adding an end of line indicator and then padding lines (eg with spaces).

This obviously works best if your file currently has a fairly uniform distribution of line lengths and doesn't have some lines that are very, very long. The downside is that this will artificially increase the size of your file.