HttpWebRequest chunked/async POST

Question

Hi I want to upload some dynamic generated content to my web api. On the client I use the HttpWebRequest. The data should be uploaded sync and I want to write to the stream

Answer 1

I found my problem.

The order of my code caused the problem.

The solution is to call it in this order:

• GetRequestStream (writing async to the stream) (the request is send to the server after the first write) then:
• GetResponseAsync()
• GetResponseStream()

My understanding is that "GetResponseAsync" triggers the client to send the request (for now the headers only), but I have discovered it to be an uneccesary step, because the request had already been sent after the first few bits had been written to the stream.

The second cause of my problems is Fiddler, but Fiddler only supports streaming of responses, and not requests.

The code acheived referencing, 'HttpWebRequest' class:

HttpWebRequest httpWebRequest = HttpWebRequest.Create("http://xxx") as HttpWebRequest;
httpWebRequest.Method = "POST";
httpWebRequest.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes("user:pw"));
httpWebRequest.PreAuthenticate = true;    
httpWebRequest.SendChunked = true;
httpWebRequest.AllowWriteStreamBuffering = false;
httpWebRequest.AllowReadStreamBuffering = false;    
httpWebRequest.ContentType = "application/octet-stream";

Stream st = httpWebRequest.GetRequestStream();

Console.WriteLine("Go");

try
{
    st.Write(buffer, 0, buffer.Length); //with the first write, the request will be send.
    st.Write(buffer, 0, buffer.Length);
    st.Write(buffer, 0, buffer.Length);

    for (int i = 1; i <= 10; i++)
    {        
        st.Write(buffer, 0, buffer.Length); //still writing while I can read on the stream at my ASP.NET web api

    }

}
catch (WebException ex)
{
    var y = ex.Response;

}
finally
{
    st.Close();

}

// Now we can read the response from the server in chunks

Task<WebResponse> response = httpWebRequest.GetResponseAsync();

Stream resultStream = response.Result.GetResponseStream();

byte[] data = new byte[1028];

int bytesRead;

while ((bytesRead = resultStream.Read(data, 0, data.Length)) > 0)
{
    string output = System.Text.Encoding.UTF8.GetString(data, 0, bytesRead);

    Console.WriteLine(output);

}

The code acheived referencing, 'HttpClient' class:

HttpClientHandler ch = new HttpClientHandler();

HttpClient c = new HttpClient(ch);    
c.DefaultRequestHeaders.TransferEncodingChunked = true;    
c.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(Encoding.ASCII.GetBytes("user:pw")));

Stream stream = new MemoryStream();

AsyncStream asyncStream = new AsyncStream(); // Custom implementation of the PushStreamContent with the method, "WriteToStream()".

PushStreamContent streamContent = new PushStreamContent(asyncStream.WriteToStream);

HttpRequestMessage requestMessage = new HttpRequestMessage(new HttpMethod("POST"), "http://XXX") { Content = streamContent };

requestMessage.Headers.TransferEncodingChunked = true;

HttpResponseMessage response = await c.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead);

// The request has been sent, since the first write in the "WriteToStream()" method.

response.EnsureSuccessStatusCode();

Task<Stream> result = response.Content.ReadAsStreamAsync();

byte[] data = new byte[1028];

int bytesRead;

while ((bytesRead = await result.Result.ReadAsync(data, 0, data.Length)) > 0)
{
    string output = System.Text.Encoding.UTF8.GetString(data, 0, bytesRead);

    Console.WriteLine(output);

}

Console.ReadKey();

Answer 2

You are using objects of an HttpWebRequest on multiple threads concurrently. response is a task that runs concurrently with your writes. That is clearly thread-unsafe.

Also, I don't see what you want to achieve. HTTP has a request-then-response model. The server can't (usually) send a single response byte before receiving the entire request. Theoretically, this is possible. But that is very unusual and likely not supported by the .NET BCL. This would only be supported by a (very unusual) custom HTTP server of yours.