How can I send MailMessages using multiple threads?

Question

I hope you guys will bear with me for my total lack of direction in case of Threading. I have to implement a Mail Queue Processing System where I have to send emails queued

Answer 1

You need to fetch messages in to memory in one place, and then route them to separate threads. I guess

Answer 2

Since sending emails is an I/O bound process, so spawning threads to send emails won't achieve much (if any) speedup.

If you're using the SMTP server that's part of Windows then when you "send" an email it doesn't actually get sent at that instant. It sits in a queue on the server and the server sends them as fast as it can. Sending emails is actually a slow process.

I guess what I'm saying is there are two options:

1. Just send them sequentially and see if that meets your performance requirements.
2. You could use a parallel programming concept called "Data Parallel" I've exampled it with examples in an blog post Data Parallel – Parallel Programming in C#/.NET

Basically, what you're doing is, you'll get all of your data (in one go). The reason is getting data in batches will also slow your process down so if you're interested in performance (which is why I'm guessing you're attempting to use threads), then don't do multiple rounds trips to the database server (which is also I/O bound on two levels, network I/O as well as disk I/O).

So get your data, and split it into chunks or partitions. This is all explained in the article I pointed to. The naive implementation would be the number of chunks equals the number of cores on the machine.

Each chunck is processed by one thread. When all threads are done, you're done. Withthe new features of the ThreadPool in .NET 4.0 (if you use Parallel.For or PLINQ or Tasks) you'll get some other benefits such as "work stealing" to further speed up the work.

Parallel.For/Parallel.ForEach will work well for you I'd think.

EDIT

Just noticed the .NET 3.5 requirement. Well the concepts still apply, but you don't have Parallel.For/ForEach. So here is an implementation (modified from my blog post) that uses the ThreadPool and using a Data Parallel technique.

    private static void SendEmailsUsingThreadPool(List<Recipient> recipients)
    {
      var coreCount = Environment.ProcessorCount;
      var itemCount = recipients.Count;
      var batchSize = itemCount / coreCount;

      var pending = coreCount;
      using (var mre = new ManualResetEvent(false))
      {
        for (int batchCount = 0; batchCount < coreCount; batchCount++)
        {
          var lower = batchCount * batchSize;
          var upper = (batchCount == coreCount - 1) ? itemCount : lower + batchSize;
          ThreadPool.QueueUserWorkItem(st =>
          {
            for (int i = lower; i < upper; i++)
              SendEmail(recipients[i]);
            if (Interlocked.Decrement(ref pending) == 0)
              mre.Set();
          });
        }
        mre.WaitOne();
      }      
    }

    private static void SendEmail(Recipient recipient)
    {
      //Send your Emails here
    }
  }

  class Recipient
  {
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string EmailAddress { get; set; }
  }

So, get your data and call SendEmailUsingThreadPool() passing it your data. Of course don't call your method that :). If you have a DataSet/DataTable then simply modify the implementation to accept a DataSet/DataTable. This methods takes care of partioning your data into chunks so you don't have to worry about any of that. Simply call it.