Why does sending post data with WebRequest take so long?

Question

I am currently creating a C# application to tie into a php / MySQL online system. The application needs to send post data to scripts and get the response.

When I send the following data

username=test&password=test  

I get the following responses...

Starting request at 22/04/2010 12:15:42  
Finished creating request : took 00:00:00.0570057  
Transmitting data at 22/04/2010 12:15:42  
Transmitted the data : took 00:00:06.9316931       <<--
Getting the response at 22/04/2010 12:15:49  
Getting response 00:00:00.0360036  
Finished response 00:00:00.0360036  
Entire call took 00:00:07.0247024  

As you can see it is taking 6 seconds to actually send the data to the script, I have done further testing bye sending data from telnet and by sending post data from a local file to the url and they dont even take a second so this is not a problem with the hosted script on the site.

Why is it taking 6 seconds to transmit the data when it is two simple strings?

I use a custom class to send the data

class httppostdata
{
    WebRequest request;
    WebResponse response;

    public string senddata(string url, string postdata)
    {
        var start = DateTime.Now;
        Console.WriteLine("Starting request at " + start.ToString());

        // create the request to the url passed in the paramaters
        request = (WebRequest)WebRequest.Create(url);


        // set the method to post
        request.Method = "POST";
        // set the content type and the content length
        request.ContentType = "application/x-www-form-urlencoded";
        request.ContentLength = postdata.Length;
        // convert the post data into a byte array
        byte[] byteData = Encoding.UTF8.GetBytes(postdata);
        var end1 = DateTime.Now;
        Console.WriteLine("Finished creating request : took " + (end1 - start));

        var start2 = DateTime.Now;
        Console.WriteLine("Transmitting data at " + start2.ToString());
        // get the request stream and write the data to it
        Stream dataStream = request.GetRequestStream();
        dataStream.Write(byteData, 0, byteData.Length);
        dataStream.Close();
        var end2 = DateTime.Now;
        Console.WriteLine("Transmitted the data : took " + (end2 - start2));


        // get the response
        var start3 = DateTime.Now;
        Console.WriteLine("Getting the response at " + start3.ToString());


        response = request.GetResponse();
        //Console.WriteLine(((WebResponse)response).StatusDescription);
        dataStream = response.GetResponseStream();
        StreamReader reader = new StreamReader(dataStream);
        var end3 = DateTime.Now;
        Console.WriteLine("Getting response " + (end3 - start3));

        // read the response
        string serverresponse = reader.ReadToEnd();
        var end3a = DateTime.Now;
        Console.WriteLine("Finished response " + (end3a - start3));

        Console.WriteLine("Entire call took " + (end3a - start));

        //Console.WriteLine(serverresponse);
        reader.Close();
        dataStream.Close();
        response.Close();

        return serverresponse;
    }
}

And to call it I use

private void btnLogin_Click(object sender, EventArgs e)
{
    // string postdata;

    if (txtUsername.Text.Length < 3 || txtPassword.Text.Length < 3)
    {
        MessageBox.Show("Missing your username or password.");
    }
    else
    {
        string postdata = "username=" + txtUsername.Text +
                          "&password=" + txtPassword.Text;

        httppostdata myPost = new httppostdata();
        string response = myPost.senddata("http://www.domainname.com/scriptname.php", postdata);
        MessageBox.Show(response);
    }
}

Answer 1

Make sure you explicitly set the proxy property of the WebRequest to null or it will try to autodetect the proxy settings which can take some time.

Answer 2

Chances are that because, in your test, you only call this once, the delay you see is the C# code being JIT compiled.

A better test would be to call this twice, and discard the timings from the first time and see if they are better.

An even better test would be to discard the first set of timings, and then run this many times and take an average, although for a very loose "indicative" view, this is probably not necessary.

As an aside, for this sort of timing, you are better off using the System.Diagnostics.Stopwatch class over System.DateTime.

[EDIT] Also, noting Mant101's suggestion about proxies, if the setting no proxy fails to resolve things, you may wish to set up Fiddler and set your request to use Fiddler as its proxy. This would allow you to intercept the actual http calls so you can get a better breakdown of the http call timings themselves from outside the framework.