I have run into an issue which can be replicated in the following way (you need IIS8 so must be on Windows 8+ or Windows Server 2012 R2+):
Create a new website in IIS Ma
As I had the same problem, here's the solution I figured out:
In your IHttpHandler you should have an static object which inherits IStopListeningRegisteredObject. Then use HostingEnvironment.RegisterObject(this) to get notified (via StopListening) when the Application Pool is recyled.
You'll also need a CancellationTokenSource (static, too), which you'll hand over in ReceiveAsync. In StopListening() you can then use Cancel() of the CancellationTokenSource to end the await. Then catch the OperationCanceledException and call Abort() on the socket.
Don't forget the Dispose() after the Cancel() or the App-Pool will still wait.