Fast and efficient way to read a space separated file of numbers into an array?

Question

I need a fast and efficient method to read a space separated file with numbers into an array. The files are formatted this way:

4 6
1 2 3 4 5 6
2 5 4 3 21111 101         


        

Answer 1

you want to stream the file into memory and parse as you go.

private IEnumerable StreamAsSpaceDelimited(this StreamReader reader)
{
    StringBuilder builder = new StringBuilder();
    int v;
    while((v = reader.Read()) != -1)
    {
        char c = (char) v;
        if(Char.IsWhiteSpace(c))
        {
            if(builder.Length >0)
            {
                yield return builder.ToString();
                builder.Clear();
            }
        }
        else
        {
            builder.Append(c);
        }
    }
    yield break;
}

this will parse the file into a collection of space delimited strings (lazily) and then you can read them as doubles just like :

using(StreamReader sr = new StreamReader("filename"))
{
    var nums = sr.StreamAsSpaceDelimited().Select(s => int.Parse(s));
    var enumerator = nums.GetEnumerator();
    enumerator.MoveNext();
    int numRows = enumerator.Current;
    enumerator.MoveNext();
    int numColumns = enumerator.current;
    int r =0, c = 0;
    int[][] destArray = new int[numRows][numColumns];
    while(enumerator.MoveNext())
    {
        destArray[r][c] = enumerator.Current;
        c++;
        if(c == numColumns)
        {
            c = 0;
            r++;
            if(r == numRows)
               break;//we are done
        }
    }

because we use iterators this should never read more than a few chars at a time. this is a common approach used to parse large files (for example this is how LINQ2CSV works).