How to stop outbound HTTP connections from timing out

问题

Backgound:

I'm currently hosting an ASP.NET application in Azure with the following specs:

ASP .Net Core 2.2
Using Flurl for HTTP requests
Kestrel Webserver
Docker (Linux - mcr.microsoft.com/dotnet/core/aspnet:2.2 runtime)
Azure App Service on P2V2 tier app service plan

I have a a couple of background jobs that run on the service that makes a lot of outbound HTTP calls to a 3rd party service.

Issue:

Under a small load (approximately 1 call per 10 seconds), all requests are completed in under a second with no issue. The issue I'm having is that under a heavy load, when service can make up to 3/4 calls in a 10 second span, some of the requests will randomly timeout and throw an exception. When I was using RestSharp the exception would read "The operation has timed out". Now that I'm using Flurl, the exception reads "The call timed out".

Here's the kicker - If I run the same job from my laptop running Windows 10 / Visual Studios 2017, this problem does NOT occur. This leads me to believe I'm hitting some limit or running out of some resource in my hosted environment. Unclear if that is connection/socket or thread related.

Things I've tried:

Ensure all code paths to the request are using async/await to prevent lockouts
Ensure Kestrel Defaults allow unlimited connections (it does by default)
Ensure Dockers default connection limits are sufficient (2000 by default, more than enough)
Configuring ServicePointManager settings for connection limits

Here is the code in my startup.cs that I'm currently using to try and prevent this issue:

public class Startup
{
    public Startup(IHostingEnvironment hostingEnvironment)
    {
        ...

        // ServicePointManager setup
        ServicePointManager.UseNagleAlgorithm = false;
        ServicePointManager.Expect100Continue = false;
        ServicePointManager.DefaultConnectionLimit = int.MaxValue;
        ServicePointManager.EnableDnsRoundRobin = true;
        ServicePointManager.ReusePort = true;

        // Set Service point timeouts
        var sp = ServicePointManager.FindServicePoint(new Uri("https://placeholder.thirdparty.com"));
        sp.ConnectionLeaseTimeout = 15 * 1000; // 15 seconds  

        FlurlHttp.ConfigureClient("https://placeholder.thirdparty.com", cli => cli.Settings.ConnectionLeaseTimeout = new TimeSpan(0, 0, 15));
    }
}

Has anyone else run into a similar issue to this? I'm open to any suggestions on how to best debug this situation, or possible methods to correct the issue. I'm at a complete loss after researching this for several days.

Thank you in advance.

回答1:

I had similar issues. Take a look at Asp.net Core HttpClient has many TIME_WAIT or CLOSE_WAIT connections . Debugging via netstat helped identify the problem for me. As one possible solution. I suggest you use IHttpClientFactory. You can get more info from https://docs.microsoft.com/en-us/aspnet/core/fundamentals/http-requests?view=aspnetcore-2.2 It should be fairly easy to use as described in Flurl client lifetime in ASP.Net Core 2.1 and IHttpClientFactory

来源：https://stackoverflow.com/questions/56386560/how-to-stop-outbound-http-connections-from-timing-out

标签

azure

Docker

asp.net-core

kestrel

flurl