Search - Articles
DevASP.NET for ASP.NET, VB.NET, XML and C# (C-Sharp) Developers Tuesday, March 03, 2009
Dev Articles
Search Directory
ASP.NET
VB.Net
C-Sharp
SQL Server
 

How to read remote web page contents in ASP.NET

Author: Jeff Nava
Download Source Code : 1395_ReadWebPage.zip

In this simple article, you will learn how to read remote web page contents in ASP.NET.

 
We can see web page HTML contents of a web page by clicking on view source or view page source in browser. You might need to get web page contents in your ASP.NET code. You can get HTML source of a page by using ASP.NET classes. This can be achieved using different techniques. You can use WebRequest and WebResponse classes to along with StreamReader class to read remote web page contents. WebRequest class has a Create() method which takes the URL of a page and initialize a new instance of WebRequest class for specified URL. GetResponse() method of WebRequest class returns a response and we can assign this to WebResponse class object. Then we can use StreamReader class to read contents from WebResponse object.  

You can also use a very simple technique to get page contents by using WebClient class. You can get byte data of page content by using DownloadData() method and you can also call DownloadString() method by providing page URL to it to directly get the HTML page contents of that page.

 

  1. Open Visual Studio 2010
  2. File > New > Website
  3. Visual basic or Visual C# > ASP.NET Empty Web Site
  4. Website > Add New Item > Web Form
  5. Add code below in aspx page

    <h3>Enter URL and get contents of the page</h3>
    <asp:TextBox ID="TextBoxURL" runat="server" Height="20px" Width="250px"></asp:TextBox>
    <asp:Button ID="Button1" runat="server" Text="Get Contents"
        onclick="Button1_Click" />
    <br />
    <br />
    <asp:TextBox ID="TextBoxPageContents" runat="server" TextMode="MultiLine"
        Height="500px" Width="1000px"></asp:TextBox>
     
  6. Now open code behind file and Include following namespaces

    C#
    using System.Net;
    using System.IO;
    using System.Text;
     
     
    VB.NET

    Imports System.Net
    Imports System.IO
    Imports System.Text
     
  7. Add code below in Button Click event

    C#
     
    protected void Button1_Click(object sender, EventArgs e)
    {
        string url = TextBoxURL.Text;
     
        WebRequest webReq = WebRequest.Create(url);
        WebResponse webRes = webReq.GetResponse();
     
        using (StreamReader reader = new StreamReader(webRes.GetResponseStream()))
        {
            TextBoxPageContents.Text = reader.ReadToEnd();
        }
    }
     
    VB.NET
     
    Protected Sub Button1_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim url As String = TextBoxURL.Text
     
        Dim webReq As WebRequest = WebRequest.Create(url)
        Dim webRes As WebResponse = webReq.GetResponse()
     
        Using reader As New StreamReader(webRes.GetResponseStream())
            TextBoxPageContents.Text = reader.ReadToEnd()
        End Using
    End Sub
     
    Here I have used WebRequest and Web Response classes to read remote web page content. 
     
  8. Now comment above code in Button click event and write code below in it

    C#
     
    protected void Button1_Click(object sender, EventArgs e)
    {
        string url = TextBoxURL.Text;
     
        WebClient client = new WebClient();
        //TextBoxPageContents.Text = client.DownloadString(url);
     
        byte[] byteData = null;
        byteData = client.DownloadData(url);
     
        UTF8Encoding UTF8Encod = new UTF8Encoding();
        TextBoxPageContents.Text = UTF8Encod.GetString(byteData);
    }
     
    VB.NET

    Protected Sub Button1_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim url As String = TextBoxURL.Text
     
        Dim client As New WebClient()
        'TextBoxPageContents.Text = client.DownloadString(url)
     
        Dim byteData() As Byte
        byteData = client.DownloadData(url)
     
        Dim UTF8Encod As UTF8Encoding = New UTF8Encoding()
        TextBoxPageContents.Text = UTF8Encod.GetString(byteData)
     
    End Sub
     
    Here I have used WebClient class to read remote page contents. I have used two method of WebClient class here. First I have called the DownloadString() method to get directly HTML page contents as string. Second I have used DownloadData() method to get byte data and then I have converted this byte data to string using UTF8Encoding class.
     
  9. Now see web site in browser. Provide URL of a page and click on button to read contents of that page. 
   
Add Article Comment:
Name :
Email Address :
   
Comments :
 
   
<< How to use multiple DataKeyNames in GridView in ASP.NET

Disclaimer - Privacy
© 2002-2017 DevASP.net