String Builder vs Lists

Question

I am reading in multiple files in with millions of lines and I am creating a list of all line numbers that have a specific issue. For example if a specific field is left bla

Answer 1

My end goal is to out put a message like "Specific field is blank on 1-32, 40, 45, 47, 49-51

If that's the end goal, no point in going through an intermediary representation such as a List<int> - just go with a StringBuilder. You will save on memory and CPU that way.

Answer 2

StringBuilder serves your purpose so stick with that, if you ever need the line numbers you can easily change the code then.

Answer 3

Is your output supposed to be human readable? If so, you'll hit the limit of what is reasonable to read, long before you have any performance/memory issues from your data structure. Use whatever is easiest for you to work with.

If the output is supposed to be machine readable, then that output might suggest an appropriate data structure.

Answer 4

As others have pointed out, I would probably use StringBuilder. The List may have to resize many times; the new implementation of StringBuilder does not have to resize.

Answer 5

Depends on how you can / want to break the code up.

Given you are reading it in line order, not sure you need a list at all. Your current desired output implies that you can't output anything until the file is completely scanned. The size of the file suggests a one pass`analysis phase would be a good idea as well, given you are going to use buffered input as opposed to reading the entire thing into memory.

I'd be tempted with an enum to describe the issue e.g Field??? is blank and then use that as the key a dictionary of string builders.

As a first thought anyway