Spawn Multiple Threads for work then wait until all finished

Question

just want some advice on \"best practice\" regarding multi-threading tasks.

as an example, we have a C# application that upon startup reads data from various \"type

Answer 1

I like @Reed's solution. Another way to accomplish the same in .NET 4.0 would be to use a CountdownEvent.

class Program
{
    static void Main(string[] args)
    {
        var numThreads = 10;
        var countdownEvent = new CountdownEvent(numThreads);

        // Start workers.
        for (var i = 0; i < numThreads; i++)
        {
            new Thread(delegate()
            {
                Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
                // Signal the CountdownEvent.
                countdownEvent.Signal();
            }).Start();
        }

        // Wait for workers.
        countdownEvent.Wait();
        Console.WriteLine("Finished.");
    }
}

Answer 2

Here are two patterns for waiting on multiple parallel operations. The trick is that you have to treat your main thread as if it were one of the parallel operations as well. Otherwise, there is a subtle race condition between the signalling of completion in the worker threads and the waiting on that signal from the main thread.

int threadCount = 1;
ManualResetEvent finished = new ManualResetEvent(false);
for (int i = 0; i < NUM_WORK_ITEMS; i++)
{
  Interlocked.Increment(ref threadCount); 
  ThreadPool.QueueUserWorkItem(delegate 
  { 
      try 
      { 
           // do work 
      } 
      finally 
      { 
          if (Interlocked.Decrement(ref threadCount) == 0) finished.Set();
      } 
  }); 
}
if (Interlocked.Decrement(ref threadCount) == 0) finished.Set();
finished.WaitOne(); 

As a personal preference I like using the CountdownEvent class to do the counting for me which is available in .NET 4.0.

var finished = new CountdownEvent(1);
for (int i = 0; i < NUM_WORK_ITEMS; i++)
{
  finished.AddCount();
  ThreadPool.QueueUserWorkItem(delegate 
  { 
      try 
      { 
           // do work 
      } 
      finally 
      { 
        finished.Signal();
      } 
  }); 
}
finished.Signal();
finished.Wait(); 

The examples above use the ThreadPool, but you can swap that for whatever threading mechanism you prefer.

Answer 3

an easier method I like to use:

    private int ThreadsCount = 100; //initialize threads count
    private void button1_Click(object sender, EventArgs e)
    {   
        for (int i = 0; i < ThreadsCount; i++)
        {
            Thread t = new Thread(new ThreadStart(myMethod));
            t.IsBackground = true;
            t.Start(); 
        } 
    }

    private void myMethod()
    {
        //everytime a thread finishes executing decrease the threads count
        ThreadsCount = ThreadsCount - 1;

        if (ThreadsCount < 1)
        {
            //if all threads finished executing do whatever you wanna do here..
            MessageBox.Show("Finished Executing all threads!!!");
        }
    }

Answer 4

If you have more than 64 wait handles for an STA Thread as Mark says. you could create a list with your threads and wait for all to complete in a second loop.

//check that all threads have completed.
foreach (Thread thread in threadList)
{
     thread.Join();

}  

Answer 5

If you're not on .NET 4.0 then you can use a List<ManualResetEvent>, one for each thread and Wait for them to be Set. To wait on multiple threads you could consider using WaitAll but watch out for the limit of 64 wait handles. If you need more than this, you can just loop over them and wait for each one individually.

If you want a faster startup exprience, you probably don't need to wait for all the data to be read during startup. Just display the GUI and any information that is missing can be shown greyed out with some sort of "Updating..." icon or similar. When the information comes in, just fire an event to update the GUI. There could be many operations that the user can begin to perform even before all the data from all tables is read in.

Answer 6

Another possibility with TPL, assuming jobs is the collections of items to process, or subthreads to run:

Task.WaitAll(jobs
    .Select(job => TaskFactory.StartNew(() => /*run job*/))
    .ToArray());