Call webservice asynchronously and wait for all threads to be finished

Question

I need to call the web service several times to get data, and then put those data into my database, so I\'ve got the following code:

foreach (string v in options         


        

Answer 1

To improve the scalability (i.e., the number of request your web app can serve simultaneously), you need to reduce the number of threads your app is using for each requests. So, instead of waiting for all threads to be finished, you should use naturally asynchronous APIs, which don't block a thread while the operation is pending. More details on this can be found here.

If you can use .NET 4.5, your specific case might be improved like this:

public void Page_Load(object sender, EventArgs e)
{
    RegisterAsyncTask(new PageAsyncTask(ProcessDataAsync));
}

public async Task ProcessDataAsync()
{
    var tasks = options.Select(v => _pi.GetDataAsync(v));
    await Task.WhenAll(tasks);
    var nodes = tasks.Select(t => t.Result);
    _dbService.SaveToDb(nodes);
}

public async Task> GetDataAsync(string v)
{
    IList nodes = null;
    using (var client = new WsClient())
    {
        IEnumerable wsNodes = 
            await client.getNodesAsync(new getClassLevel { code = v });
        nodes = ProcessData(wsNodes);
    }
    return nodes;
}

The client-side proxy for WsClient probably already has an async version of getNodes, called getNodesAsync (if not, check this). ProcessDataAsync starts a bunch of parallel non-blocking GetDataAsync tasks (for each node), and asynchronously awaits their completion. That's what makes this code scale well.

You might be able to further improve ProcessDataAsync by saving the data asynchronously, i.e. await _dbService.SaveToDbAsync(nodes), if you can leverage the use of asynchronous Task-based DB API (e.g, with EF6).