How to get a Fast .Net Http Request

Question

I need an Http request that I can use in .Net which takes under 100 ms. I\'m able to achieve this in my browser so I really don\'t see why this is such a problem in code.

Answer 1

When benchmarking, it is best to discard at least the first two timings as they are likely to skew the results:

• Timing 1: Dominated by JIT overhead i.e. the process of turning byte code into native code.
• Timing 2: A possible optimization pass for the JIT'd code.

Timings after this will reflect repeat performance much better.

The following is an example of a test harness that will automatically disregard JIT and optimization passes, and run a test a set number of iterations before taking an average to assert performance. As you can see the JIT pass takes a substantial amount of time.

JIT:410.79ms

Optimize:0.98ms.

Average over 10 iterations:0.38ms

Code:

[Test]
public void WebRequest_Should_Get_Html_Quickly()
{
    private const int TestIterations = 10;
    private const int MaxMilliseconds = 100;

    Action test = () =>
    {
       WebRequest.Create("http://localhost/iisstart.htm").GetResponse();
    };

    AssertTimedTest(TestIterations, MaxMilliseconds, test);
}

private static void AssertTimedTest(int iterations, int maxMs, Action test)
{
    double jit = Execute(test); //disregard jit pass
    Console.WriteLine("JIT:{0:F2}ms.", jit);

    double optimize = Execute(test); //disregard optimize pass
    Console.WriteLine("Optimize:{0:F2}ms.", optimize);

    double totalElapsed = 0;
    for (int i = 0; i < iterations; i++) totalElapsed += Execute(test);

    double averageMs = (totalElapsed / iterations);
    Console.WriteLine("Average:{0:F2}ms.", averageMs);
    Assert.Less(averageMs, maxMs, "Average elapsed test time.");
}

private static double Execute(Action action)
{
    Stopwatch stopwatch = Stopwatch.StartNew();
    action();
    return stopwatch.Elapsed.TotalMilliseconds;
}

Answer 2

Use the StopWatch class to get accurate timings.

Then, make sure you're not seeing the results of un-optimized code or JIT compilation by running your timing test several times in Release code. Discard the first few calls to remove he impact of JIT and then take the mean tidings of the rest.

VS.NET has the ability to measure performance, and you might also want to use something like Fiddler to see how much time you're spending "on the wire" and sanity check that it's not your IIS/web server causing the delays.

500ms is a very long time, and it's possible to be in the 10s of ms with these classes, so don't give up hope (yet).

Update #1:

This is a great article that talks about micro benchmarking and what's needed to avoid seeing things like JIT:

http://blogs.msdn.com/b/vancem/archive/2009/02/06/measureit-update-tool-for-doing-microbenchmarks.aspx

You're not quite micro-benchmarking, but there are lots of best practices in here.

Update #2:

So, I wrote this console app (using VS.NET 2010)...

class Program
{
    static void Main(string[] args)
    {
        var stopwatch = Stopwatch.StartNew();
        var req = (HttpWebRequest)WebRequest.Create("http://localhost");
        var response = req.GetResponse();
        Console.WriteLine(stopwatch.ElapsedMilliseconds);            
    }
}

... and Ctrl-F5'd it. It was compiled as debug, but I ran it without debugging, and I got 63ms. I'm running this on my Windows 7 laptop, and so http://localhost brings back the default IIS7 home page. Running it again I get similar times.

Running a Release build gives times in the 50ms to 55ms range.

This is the order of magnitude I'd expect. Clearly, if your website is performing an ASP.NET recompile, or recycling the app pool, or doing lots of back end processing, then your timings will differ. If your markup is massive then it will also differ, but none of the classes you're using client side should be the rate limiting steps here. It'll be the network hope and/or the remote app processing.

Answer 3

Try setting the Proxy property of the HttpWebRequest instance to null. If that works, then try setting it to GlobalProxySelection.GetEmptyWebProxy(), which seems to be more correct.

You can read more about it here: - WebRequest slow?: http://holyhoehle.wordpress.com/2010/01/12/webrequest-slow/

---

Update 2018: Pulling this up from the comments.

System.Net.GlobalProxySelection is obsolete.This class has been deprecated. Please use WebRequest.DefaultWebProxy instead to access and set the global default proxy. Use null instead of GetEmptyWebProxy(). – jirarium Jul 22 '17 at 5:44