Get the source of some website from asp.net code

Question

Is there any way that I could get the source of a website (as a string preferably), let\'s say www.google.com, from some c# code inside code behind of asp.net website?

Answer 1

For C#, I prefer to use HttpWebRequest over WebClient because you can have more option in the future like having GET/POST parameter, using Cookies, etc.

You can have a shortest explication at MSDN.

Here is the example from MSDN:

        // Create a new HttpWebRequest object.
        HttpWebRequest request=(HttpWebRequest) WebRequest.Create("http://www.contoso.com/example.aspx");    

        // Set the ContentType property. 
        request.ContentType="application/x-www-form-urlencoded";
        // Set the Method property to 'POST' to post data to the URI.
        request.Method = "POST";
        // Start the asynchronous operation.    
        request.BeginGetRequestStream(new AsyncCallback(ReadCallback), request);    

        // Keep the main thread from continuing while the asynchronous
        // operation completes. A real world application
        // could do something useful such as updating its user interface. 
        allDone.WaitOne();

        // Get the response.
        HttpWebResponse response = (HttpWebResponse)request.GetResponse();
        Stream streamResponse = response.GetResponseStream();
        StreamReader streamRead = new StreamReader(streamResponse);
        string responseString = streamRead.ReadToEnd();
        Console.WriteLine(responseString);
        // Close the stream object.
        streamResponse.Close();
        streamRead.Close();

        // Release the HttpWebResponse.
        response.Close();

Answer 2

it's not the most obvious (and the best) way but i found out that in windows forms you can use WebBrowser control (if you actually need it), fill it's Url property with the url you need and when it's loaded, read the DocumentText property - it contains the html code of the viewed site.

Answer 3

Assuming you want to retrieve the html:

class Program
{
    static void Main(string[] args)
    {
        using (WebClient client = new WebClient())
        using (Stream stream = client.OpenRead("http://www.google.com"))
        using (StreamReader reader = new StreamReader(stream))
        {
            Console.WriteLine(reader.ReadToEnd());
        }
    }
}