Best way to test high-volume SMTP email sending code?

Question

I\'ve written a component in a Windows service (C#) which is responsible for sending sometimes large volumes of emails. These emails will go to recipients on many domains &ndas

Answer 1

After searching around, I ended up firing up Papercut on several extra machines I had laying around. I then populated my database with test addresses *@[test-machine-*.local].

While this did work well enough, I tested with 25 send threads and it looked like I was overwhelming the four computers running Papercut. Several hundred send attempts experienced TCP connection failures; those messages were properly requeued to be sent later (and ultimately did arrive). However, out of 25,000 test emails, about 500 simply disappeared – adding up the *.eml files in Papercut's folder on each test machine yielded only ~24,500.

Now I'm left wondering whether the missing emails are a due to a problem in my code, or if Papercut dropped messages which it reported in SMTP as 250 OK.

Answer 2

Assume you've got two servers available. One will be the sender, one will be the receiver. You can set up DNS (or even just hosts files) on both with a long series of fake domains. As far as the two servers are concerned, those domains are perfectly valid as the local DNS servers are authoritative for them, but are completely invalid as far as the rest of the net is concerned. Just make sure the resolver checks the hosts file before DNS.

Once that's done, you can have the sending server spam the receiving server to your heart's content, as have the receiver do various things to test your code's reactions. Greylisting, TCP delays, hard bounces, ICMP unreachables, ICMP hops exceeded, etc...

Of course, given you have to test all these conditions, you're basically creating your own SMTP server, so why not use an actual one to begin with? I'd guess the effort required to do some basic parseing of bounce messages will be far less than having to come up with code chunks to handle all the failure modes that postfix/sendmail/exim/etc... already handle perfectly well on their own.

And this is especially true when you consider your sending code has to be perfect from the get-go. If an email blast fails part-way through and only half the recipient list gets the message, you're in a far bigger hole than if a few hundred or a few thousand messages bounce. Or worse yet, fails in multiple different ways (some servers unreachable, some greylisting you for excessive traffic, etc...). Whereas bounces will happily sit in the incoming queue until you process them manually, or patch up your bounce parser to handle them.