Is it possible to mock out a .NET HttpWebResponse?

Question

i\'ve got an integration test that grabs some json result from a 3rd party server. It\'s really simple and works great.

I was hoping to stop actually hitting this se

Answer 1

If it helps please find below the code illustrated in the accepted answer using NSubstitute in place of Moq

using NSubstitute; /*+ other assemblies*/

[TestMethod]
public void Create_should_create_request_and_respond_with_stream()
{
   //Arrange
   var expected = "response content";
   var expectedBytes = Encoding.UTF8.GetBytes(expected);
   var responseStream = new MemoryStream();
   responseStream.Write(expectedBytes, 0, expectedBytes.Length);
   responseStream.Seek(0, SeekOrigin.Begin);

   var response = Substitute.For<HttpWebResponse>();
   response.GetResponseStream().Returns(responseStream);

   var request = Substitute.For<HttpWebRequest>();
   request.GetResponse().Returns(response);

   var factory = Substitute.For<IHttpWebRequestFactory>();
   factory.Create(Arg.Any<string>()).Returns(request);

   //Act
   var actualRequest = factory.Create("http://www.google.com");
   actualRequest.Method = WebRequestMethods.Http.Get;

   string actual;

   using (var httpWebResponse = (HttpWebResponse)actualRequest.GetResponse())
   {
       using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
       {
           actual = streamReader.ReadToEnd();
       }
   }

   //Assert
   Assert.AreEqual(expected, actual);
}

public interface IHttpWebRequestFactory
{
    HttpWebRequest Create(string uri);
}

Unit Test run and passes successfully.

Up vote given on the answer I've been looking for some time how to do this effectively.

Answer 2

None of the Microsoft's HTTP stack was developed with unit testing and separation in mind.

You have three options:

• Make the call to web as small as possible (i.e. send and get back data and pass to other methods) and test the rest. As far as the web call is concerned, there should be much magic happening there and very straightforward.
• Wrap the HTTP call in another class and pass your mock object while testing.
• Wrap HttpWebResponse and HttpWebRequest by two other classes. This is what the MVC team did with HttpContext.

Second option:

interface IWebCaller
{
    string CallWeb(string address);
}

Answer 3

You can actually return HttpWebResponse without mocking, see my answer here. It does not require any "external" proxying interfaces, only the "standard" WebRequest WebResponse and ICreateWebRequest.

If you don't need access to HttpWebResponse and can deal with just WebResponse it is even easier; we do that in our unit tests to return "prefabricated" content responses for consumption. I had to "go the extra mile" in order to return actual HTTP status codes, to simulate e.g. 404 responses which requires you use HttpWebResponse so you can access the StatusCode property et al.

The other solutions assuming everything is HttpWebXXX ignores everything supported by WebRequest.Create() except HTTP, which can be a handler for any registered prefix you care to use (via WebRequest.RegisterPrefix() and if you are ignoring that, you are missing out, because it is a great way to expose other content streams you otherwise have no way to access, e.g. Embeeded Resource Streams, File streams, etc.

Also, explicitly casting the return of WebRequest.Create() to HttpWebRequest is a path to breakage, since the method return type is WebRequest and again, shows some ignorance of how that API actually works.

Answer 4

I Found a great solution in this blog post:

It´s very easy to use, you just need to do this:

string response = "my response string here";
WebRequest.RegisterPrefix("test", new TestWebRequestCreate());
TestWebRequest request = TestWebRequestCreate.CreateTestRequest(response);

And copy those files to your project:

    class TestWebRequestCreate : IWebRequestCreate
{
    static WebRequest nextRequest;
    static object lockObject = new object();

    static public WebRequest NextRequest
    {
        get { return nextRequest ;}
        set
        {
            lock (lockObject)
            {
                nextRequest = value;
            }
        }
    }

    /// <summary>See <see cref="IWebRequestCreate.Create"/>.</summary>
    public WebRequest Create(Uri uri)
    {
        return nextRequest;
    }

    /// <summary>Utility method for creating a TestWebRequest and setting
    /// it to be the next WebRequest to use.</summary>
    /// <param name="response">The response the TestWebRequest will return.</param>
    public static TestWebRequest CreateTestRequest(string response)
    {
        TestWebRequest request = new TestWebRequest(response);
        NextRequest = request;
        return request;
    }
}

class TestWebRequest : WebRequest
{
    MemoryStream requestStream = new MemoryStream();
    MemoryStream responseStream;

    public override string Method { get; set; }
    public override string ContentType { get; set; }
    public override long ContentLength { get; set; }

    /// <summary>Initializes a new instance of <see cref="TestWebRequest"/>
    /// with the response to return.</summary>
    public TestWebRequest(string response)
    {
        responseStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(response));
    }

    /// <summary>Returns the request contents as a string.</summary>
    public string ContentAsString()
    {
        return System.Text.Encoding.UTF8.GetString(requestStream.ToArray());
    }

    /// <summary>See <see cref="WebRequest.GetRequestStream"/>.</summary>
    public override Stream GetRequestStream()
    {
        return requestStream;
    }

    /// <summary>See <see cref="WebRequest.GetResponse"/>.</summary>
    public override WebResponse GetResponse()
    {
        return new TestWebReponse(responseStream);
    }
}

class TestWebReponse : WebResponse
{
    Stream responseStream;

    /// <summary>Initializes a new instance of <see cref="TestWebReponse"/>
    /// with the response stream to return.</summary>
    public TestWebReponse(Stream responseStream)
    {
        this.responseStream = responseStream;
    }

    /// <summary>See <see cref="WebResponse.GetResponseStream"/>.</summary>
    public override Stream GetResponseStream()
    {
        return responseStream;
    }
}

Answer 5

You may wish to change your consuming code to take in an interface for a factory that creates requests and responses that can be mocked which wrap the actual implementation.

Update: Revisiting

I've been getting downvotes long after my answer was accepted, and I admit my original answer was poor quality and made a big assumption.

Mocking HttpWebRequest in 4.5+

The confusion from my original answer lies in the fact that you can mock HttpWebResponse in 4.5, but not earlier versions. Mocking it in 4.5 also utilizes obsolete constructors. So, the recommended course of action is to abstract the request and response. Anyways, below is a complete working test using .NET 4.5 with Moq 4.2.

[Test]
public void Create_should_create_request_and_respond_with_stream()
{
    // arrange
    var expected = "response content";
    var expectedBytes = Encoding.UTF8.GetBytes(expected);
    var responseStream = new MemoryStream();
    responseStream.Write(expectedBytes, 0, expectedBytes.Length);
    responseStream.Seek(0, SeekOrigin.Begin);

    var response = new Mock<HttpWebResponse>();
    response.Setup(c => c.GetResponseStream()).Returns(responseStream);

    var request = new Mock<HttpWebRequest>();
    request.Setup(c => c.GetResponse()).Returns(response.Object);

    var factory = new Mock<IHttpWebRequestFactory>();
    factory.Setup(c => c.Create(It.IsAny<string>()))
        .Returns(request.Object);

    // act
    var actualRequest = factory.Object.Create("http://www.google.com");
    actualRequest.Method = WebRequestMethods.Http.Get;

    string actual;

    using (var httpWebResponse = (HttpWebResponse)actualRequest.GetResponse())
    {
        using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
        {
            actual = streamReader.ReadToEnd();
        }
    }


    // assert
    actual.Should().Be(expected);
}

public interface IHttpWebRequestFactory
{
    HttpWebRequest Create(string uri);
}

Better answer: Abstract the Response and Request

Here's a safer bare-bones implementation of an abstraction that will work for prior versions (well, down to 3.5 at least):

[Test]
public void Create_should_create_request_and_respond_with_stream()
{
    // arrange
    var expected = "response content";
    var expectedBytes = Encoding.UTF8.GetBytes(expected);
    var responseStream = new MemoryStream();
    responseStream.Write(expectedBytes, 0, expectedBytes.Length);
    responseStream.Seek(0, SeekOrigin.Begin);

    var response = new Mock<IHttpWebResponse>();
    response.Setup(c => c.GetResponseStream()).Returns(responseStream);

    var request = new Mock<IHttpWebRequest>();
    request.Setup(c => c.GetResponse()).Returns(response.Object);

    var factory = new Mock<IHttpWebRequestFactory>();
    factory.Setup(c => c.Create(It.IsAny<string>()))
        .Returns(request.Object);

    // act
    var actualRequest = factory.Object.Create("http://www.google.com");
    actualRequest.Method = WebRequestMethods.Http.Get;

    string actual;

    using (var httpWebResponse = actualRequest.GetResponse())
    {
        using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
        {
            actual = streamReader.ReadToEnd();
        }
    }


    // assert
    actual.Should().Be(expected);
}

public interface IHttpWebRequest
{
    // expose the members you need
    string Method { get; set; }

    IHttpWebResponse GetResponse();
}

public interface IHttpWebResponse : IDisposable
{
    // expose the members you need
    Stream GetResponseStream();
}

public interface IHttpWebRequestFactory
{
    IHttpWebRequest Create(string uri);
}

// barebones implementation

private class HttpWebRequestFactory : IHttpWebRequestFactory
{
    public IHttpWebRequest Create(string uri)
    {
        return new WrapHttpWebRequest((HttpWebRequest)WebRequest.Create(uri));
    }
}

public class WrapHttpWebRequest : IHttpWebRequest
{
    private readonly HttpWebRequest _request;

    public WrapHttpWebRequest(HttpWebRequest request)
    {
        _request = request;
    }

    public string Method
    {
        get { return _request.Method; }
        set { _request.Method = value; }
    }

    public IHttpWebResponse GetResponse()
    {
        return new WrapHttpWebResponse((HttpWebResponse)_request.GetResponse());
    }
}

public class WrapHttpWebResponse : IHttpWebResponse
{
    private WebResponse _response;

    public WrapHttpWebResponse(HttpWebResponse response)
    {
        _response = response;
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    private void Dispose(bool disposing)
    {
        if (disposing)
        {
            if (_response != null)
            {
                ((IDisposable)_response).Dispose();
                _response = null;
            }
        }
    }

    public Stream GetResponseStream()
    {
        return _response.GetResponseStream();
    }
}

Answer 6

Rather than mocking out the HttpWebResponse is I would wrap the call behind an interface, and mock that interface.

If you are testing does the web response hit the site I want it too, that is a different test than if does class A call the WebResponse interface to get the needed data.

For mocking an interface I prefer Rhino mocks. See here on how to use it.