Efficient Method for Creating CSV String from Lists/SortedLists C#?

Question

I have an application which implements Asynchronous SOAP. Every 50-100ms I will receive data which is converted into a SortedList object. I als

Answer 1

I agree with answer from Conrad - yet one more idea to improve performance would be to do reverse lookup i.e. take each element from SortedList and do lookup in other list (of course, I would recommend to have dictionary instead of list for faster lookup).

Answer 2

I'm sure I haven't come up with the most efficient algorithm, but here is a starting point, at least. If nothing else, you will notice the use of StringBuilder, rather than concatenating strings. This alone is likely to garner you some performace benefit.

This algorithm assumes that both the SortedList keys and the "data" List are ordered in the same way (low-to-high).

var textBuilder = new StringBuilder(timestamp.ToString("HH:mm:ss"));

var index = 0;
foreach(double key in data.Keys)
{
    while(Allkeys[index] < key)
    {
        textBuilder.Append(",0.0");
        index++;
    }

    textBuilder.Append(",").Append(data[key]);
    index++;
}
MyText = textBuilder.Append(@"\n").ToString();

Just looking at the above, I'm sure there is a bug, but not sure what or where without spending more time and/or testing.

A possible LINQ solution is more declarative:

var textBuilder = new StringBuilder(timestamp.ToString("HH:mm:ss"));

var values = Allkeys.Select(
    key => data.ContainsKey(key) ? data[key].ToString() : "0.0")
    .ToArray();

var data = String.Join(",", values);
var MyText = textBuilder.Append(data).Append(@"\n").ToString();

More can be included in the LINQ expression using the Aggregate extension method, but you'd have to use string concatenation in the accumulator, so I haven't shown that here.

Answer 3

Here are some thoughts.

1) Use a StreamWriter to write instead of File. This will be faster than the two step of writing to memory and then to a file.

2) If it all possible parallelize the work. For example if you can you one thread for processing the message and another thread to write the message.

3) I don't think the intent of LINQ is to improve performance, but make manipulations of data easier