C# Downloader: should I use Threads, BackgroundWorker or ThreadPool?

Question

I\'m writing a downloader in C# and stopped at the following problem: what kind of method should I use to parallelize my downloads and update my GUI?

In my first att

Answer 1

Having 100% cpu load has nothing to do with the download (as your network is practically always the bottleneck). I would say you have to check your logic how you wait for the download to complete.

Can you post some code of the thread's code you start multiple times?

Answer 2

I would suggest using WebClient.DownloadFileAsync for this. You can have multiple downloads going, each raising the DownloadProgressChanged event as it goes along, and DownloadFileCompleted when done.

You can control the concurrency by using a queue with a semaphore or, if you're using .NET 4.0, a BlockingCollection. For example:

// Information used in callbacks.
class DownloadArgs
{
    public readonly string Url;
    public readonly string Filename;
    public readonly WebClient Client;
    public DownloadArgs(string u, string f, WebClient c)
    {
        Url = u;
        Filename = f;
        Client = c;
    }
}

const int MaxClients = 4;

// create a queue that allows the max items
BlockingCollection<WebClient> ClientQueue = new BlockingCollection<WebClient>(MaxClients);

// queue of urls to be downloaded (unbounded)
Queue<string> UrlQueue = new Queue<string>();

// create four WebClient instances and put them into the queue
for (int i = 0; i < MaxClients; ++i)
{
    var cli = new WebClient();
    cli.DownloadProgressChanged += DownloadProgressChanged;
    cli.DownloadFileCompleted += DownloadFileCompleted;
    ClientQueue.Add(cli);
}

// Fill the UrlQueue here

// Now go until the UrlQueue is empty
while (UrlQueue.Count > 0)
{
    WebClient cli = ClientQueue.Take(); // blocks if there is no client available
    string url = UrlQueue.Dequeue();
    string fname = CreateOutputFilename(url);  // or however you get the output file name
    cli.DownloadFileAsync(new Uri(url), fname, 
        new DownloadArgs(url, fname, cli));
}


void DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
    DownloadArgs args = (DownloadArgs)e.UserState;
    // Do status updates for this download
}

void DownloadFileCompleted(object sender, AsyncCompletedEventArgs e)
{
    DownloadArgs args = (DownloadArgs)e.UserState;
    // do whatever UI updates

    // now put this client back into the queue
    ClientQueue.Add(args.Client);
}

There's no need for explicitly managing threads or going to the TPL.

Answer 3

By creating 4 different backgroundworkers you will be creating seperate threads that will no longer interfere with your GUI. Backgroundworkers are simple to implement and from what I understand will do exactly what you need them to do.

Personally I would do this and simply allow the others to not start until the previous one is finished. (Or maybe just one, and allow it to execute one method at a time in the correct order.)

FYI - Backgroundworker

Answer 4

I think you should look into using the Task Parallel Library, which is new in .NET 4 and is designed for solving these types of problems