getting argument exception in concurrent dictionary when sorting and displaying as it is being updated

Question

I am getting a hard to reproduce error in the following program in which a number of threads update a concurrent dictionary in parallel and the main thread displays the stat

Answer 1

After a discussion with ChrisShain in the comments, the conclusion is that you should get mutually exclusive access to the dictionary before printing it out, either with a mutex of a lock statement.

Doing it with a lock:

public void WriteBatch(IEnumerable<Tuple<string, int>> batch)
{
    lock (myLock) 
    {
        foreach (var tuple in batch)
        {
            Console.WriteLine("{0} - {1}", tuple.Item1, tuple.Item2);
        }
        Console.WriteLine();
    }
}

assuming you allocated a myLock object at the class level. See example.

Doing it with a mutex:

public void WriteBatch(IEnumerable<Tuple<string, int>> batch)
{
    mut.WaitOne();

    foreach (var tuple in batch)
    {
        Console.WriteLine("{0} - {1}", tuple.Item1, tuple.Item2);
    }
    Console.WriteLine();

    mut.ReleaseMutex();
}

Again, assuming you allocated a Mutex object at the class level. See example.

Answer 2

As others have said, there is a race in the constructor of the internal class System.Linq.Buffer<T>, which is called by OrderBy.

Here is the offending code snippet:

TElement[] array = null;
int num = 0;
if (collection != null)
{
    num = collection.Count;
    if (num > 0)
    {
        array = new TElement[num];
        collection.CopyTo(array, 0);
    }
}

The exception is thrown when item(s) are added to the collection after the call to collection.Count but before the call to collection.CopyTo.

---

As a work around, you can make a "snapshot" copy of the dictionary before you sort it.

You can do this by calling ConcurrentDictionary.ToArray.
As this is implemented in the ConcurrentDictionary class itself, it is safe.

Using this approach means you don't have to protect the collection with a lock which, as you say, defeats the purpose of using a concurrent collection in the first place.

while (!completed)
{
    completed = t.Join(1);

    var q =
      from pair in wordFrequencies.ToArray() // <-- add ToArray here
      orderby pair.Value descending, pair.Key
      select new Tuple<string, int>(pair.Key, pair.Value);

    outputter.WriteBatch(q);
}