async and await: are they bad?

Question

We recently developed a site based on SOA but this site ended up having terrible load and performance issues when it went under load. I posted a question related this issue

Answer 1

There is a lot of stuff to discuss.

First of all, async/await can help you naturally when your application has almost no business logic. I mean the point of async/await is to do not have many threads in sleep mode waiting for something, mostly some IO, e.g. database queries (and fetching). If your application does huge business logic using cpu for 100%, async/await does not help you.

The problem of 900 threads is that they are inefficient - if they run concurrently. The point is that it's better to have such number of "business" threads as you server has cores/processors. The reason is thread context switching, lock contention and so on. There is a lot of systems like LMAX distruptor pattern or Redis which process data in one thread (or one thread per core). It's just better as you do not have to handle locking.

How to reach described approach? Look at disruptor, queue incoming requests and processed them one by one instead of parallel.

Opposite approach, when there is almost no business logic, and many threads just waits for IO is good place where to put async/await into work.

How it mostly works: there is a thread which reads bytes from network - mostly only one. Once some some request arrive, this thread reads the data. There is also limited thread pool of workers which processes requests. The point of async is that once one processing thread is waiting for some thing, mostly io, db, the thread is returned in poll and can be used for another request. Once IO response is ready, some thread from pool is used to finish the processing. This is the way how you can use few threads to server thousand request in a second.

I would suggest that you should draw some picture how your site is working, what each thread does and how concurrently it works. Note that it's necessary to decide whether throughput or latency is important for you.

Answer 2

Is there anything silly with this method? Note that when we converted all method to non-async methods we got a heaps better performance.

I can see at least two things going wrong here:

public static async Task<T> GetApiResponse<T>(object parameters, string action, CancellationToken ctk)
{
        using (var httpClient = new HttpClient())
        {
            httpClient.BaseAddress = new Uri(BaseApiAddress);

            var formatter = new JsonMediaTypeFormatter();

            return
                await
                    httpClient.PostAsJsonAsync(action, parameters, ctk)
                        .ContinueWith(x => x.Result.Content
                            .ReadAsAsync<T>(new[] { formatter }).Result, ctk);
        }
    }

Firstly, the lambda you're passing to ContinueWith is blocking:

x => x.Result.Content.ReadAsAsync<T>(new[] { formatter }).Result

This is equivalent to:

x => { 
    var task = x.Result.Content.ReadAsAsync<T>(new[] { formatter });
    task.Wait();
    return task.Result;
};

Thus, you're blocking a pool thread on which the lambda is happened to be executed. This effectively kills the advantage of the naturally asynchronous ReadAsAsync API and reduces the scalability of your web app. Watch out for other places like this in your code.

Secondly, an ASP.NET request is handled by a server thread with a special synchronization context installed on it, AspNetSynchronizationContext. When you use await for continuation, the continuation callback will be posted to the same synchronization context, the compiler-generated code will take care of this. OTOH, when you use ContinueWith, this doesn't happen automatically.

Thus, you need to explicitly provide the correct task scheduler, remove the blocking .Result (this will return a task) and Unwrap the nested task:

return
    await
        httpClient.PostAsJsonAsync(action, parameters, ctk).ContinueWith(
            x => x.Result.Content.ReadAsAsync<T>(new[] { formatter }), 
            ctk,
            TaskContinuationOptions.None, 
            TaskScheduler.FromCurrentSynchronizationContext()).Unwrap();

That said, you really don't need such added complexity of ContinueWith here:

var x = await httpClient.PostAsJsonAsync(action, parameters, ctk);
return await x.Content.ReadAsAsync<T>(new[] { formatter });

The following article by Stephen Toub is highly relevant:

"Async Performance: Understanding the Costs of Async and Await".

If I have to call an async method in a sync context, where using await is not possible, what is the best way of doing it?

You almost never should need to mix await and ContinueWith, you should stick with await. Basically, if you use async, it's got to be async "all the way".

For the server-side ASP.NET MVC / Web API execution environment, it simply means the controller method should be async and return a Task or Task<>, check this. ASP.NET keeps track of pending tasks for a given HTTP request. The request is not getting completed until all tasks have been completed.

If you really need to call an async method from a synchronous method in ASP.NET, you can use AsyncManager like this to register a pending task. For classic ASP.NET, you can use PageAsyncTask.

At worst case, you'd call task.Wait() and block, because otherwise your task might continue outside the boundaries of that particular HTTP request.

For client side UI apps, some different scenarios are possible for calling an async method from synchronous method. For example, you can use ContinueWith(action, TaskScheduler.FromCurrentSynchronizationContext()) and fire an completion event from action (like this).

Answer 3

async and await should not create a large number of threads, particularly not with just 16 users. In fact, it should help you make better use of threads. The purpose of async and await in MVC is to actually give up the thread pool thread when it's busy processing IO bound tasks. This suggests to me that you are doing something silly somewhere, such as spawning threads and then waiting indefinitely.

Still, 900 threads is not really a lot, and if they're using 100% cpu, then they're not waiting.. they're chewing on something. It's this something that you should be looking into. You said you have used tools like NewRelic, well what did they point to as the source of this CPU usage? What methods?

If I were you, I would first prove that merely using async and await are not the cause of your problems. Simply create a simple site that mimics the behavior and then run the same tests on it.

Second, take a copy of your app, and start stripping stuff out and then running tests against it. See if you can track down where the problem is exactly.