Async HttpWebRequest with no wait from within a web application

Question

In my web application (ASP.NET) I have a block of code that uses HttpWebRequest to make a call to a REST service and continue execution. Right now it\'s taking longer than I

Answer 1

you are probably looking at "fire and forget" pattern. Here are some links.

http://weblogs.asp.net/albertpascual/archive/2009/05/14/fire-and-forget-class-for-asp-net.aspx

http://haacked.com/archive/2009/01/09/asynchronous-fire-and-forget-with-lambdas.aspx

http://www.eggheadcafe.com/articles/20060727.asp

hope this helps

Answer 2

(Note: I started to enter this as a comment, but as I gained some extra knowledge while researching, it grew big enough for an answer)

1. ThreadPool.QueueUserWorkItem

The first two links that @ram provides, as well as @Bryan Batchelders answer make use of the controversial ThreadPool.QueueUserWorkItem. It's controversial within the context of ASP.NET because uncareful use may starve your thread pool, as @Perhentian link shows.

2. BeginInvoke

Then I had a look at @ram's third link, which makes use of BeginInvoke. In its essence, this just seems to tell some code to run on another thread too. So no resolution here.

3. BeginGetResponse

Now back to @Perhentians link. It states how BeginGetResponse is somewhat different from an actual thread, because it uses IO Completion Ports (IOPCs). So what you're actually looking for, is a solution which still uses these IOPCs without creating an extra thread.

Now, in order to see how .NET does this, I tried to dig into HttpWebRequest.BeginGetResponse, which is really a rag of internal calls. It goes like:

1. HttpWebRequest.BeginGetResponse
2. ServicePoint.SubmitRequest
3. Connection.SubmitRequest
4. Two options:
   • If the connection is clear: Connection.StartRequest
   • Otherwise: Connection.WaitList.Add(request)

A. WaitList

First lets consider option 2: the said WaitList gets processed when a connection is done with a previous request. Taking a look at the ConnectStream involved in this whole chain shows a ThreadPool.QueueUserWorkItem with the remark:

// otherwise we queue a work item to parse the chunk
// Consider: Will we have an issue of thread dying off
// if we make a IO Read from that thread???

So, at least in some fallback scenario's, threads can still inadvertently get spawned by the framework, by just using BeginGetResponse!

B. Connection.StartRequest

Now, we still have the scenario where the connection is clear. The callback is installed by System.Net.Sockets.Socket.BeginConnect and actually invoked by BeginAccept. Some more digging reveals a call to ThreadPool.UnsafeRegisterWaitForSingleObject whose result is used to wait for.

Conclusion

Finally, we can tell what's actually going on when doing a BeginGetResponse:

// 1. Connecting a socket
UnsafeNclNativeMethods.OSSOCK.WSAConnect(m_handle)

// 2. Registering the callback
m_RegisteredWait = ThreadPool.UnsafeRegisterWaitForSingleObject(m_AsyncEvent, s_RegisteredWaitCallback, this, Timeout.Infinite, true);

// 3. Waiting for the socket to complete
UnsafeNclNativeMethods.OSSOCK.WSAEventSelect(m_handle, m_AsyncEvent.SafeWaitHandle, blockEventBits);

Note the Timeout.Infinite. I'm still investigating whether it is possible to get the socket running a codepath that does not perform the wait, but for now, it looks impossible to me.

As goes for my conclusion: there seems to be no easy way to use IOCPs in a Fire-And-Forget scenario. So unless you can get the above socket to not wait for completion on BeginAccept, you're stuck with ThreadPool.

Answer 3

HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create(myUrl);
//set up web request...
ThreadPool.QueueUserWorkItem(o=>{ myRequest.GetResponse(); });

Also known as Fire-and-Forget.