VB.Net (or C#) 2008 Multi Threaded Import

Question

I am looking to build a multi-threaded text import facility (generally CSV into SQL Server 2005) and would like to do this in VB.NET but I am not against C#. I have VS 2008 tr

Answer 1

About the best threading document I ever found was this http://www.albahari.com/threading/

If I may, the problem with simple examples is that that they're often too simple. Once you get past the counting or sort in background demos you generally need to update the UI or similar and there are some gotchas. Similarly you rarely have to deal with resource contention in simple examples and having threads degrade gracefully when a resource isn't available (such as a Db connection) requires thought.

Conceptually you need to decide how you're going to distribute your work across the threads and how many do you want. There's overhead associated with managing threads and some mechanisms use a shared thread pool that could be subject to resource contention itself (for example, any time you run a program that simply displays an empty form, how many threads do you see under task manager).

So for your case, you threads doing the actual uploading need to signal back if they've completed, if they've failed (and what the failure was). The controller needs to be able to deal with those and manage the start/stop processes and so on.

Finally (almost), assuming that making something multithread will increase performance doesn't always hold true. If for example, you chop a file up into segments but it has to travel across a low speed link (ADSL say), you're constrained by external forces and no amount of threading trickery is going to get around that. The same can apply for database updates, web requests, anything invloving large amounts of disk i/o and so on.

Despite all this, I'm not the prophet of doom. The references here are more than adequate to help you achieve what you want but be aware that one of the reasons threading seems complicated is because it can be :)

If you want more control than the BackgroundWorker/Threadpool but don't want to do everything yourself there are at least two very good freebie threading libraries knocking around the place (Wintellect & PowerThreading)

Cheers

Simon