Reading Web Pages using Excel VBA

Question

I want to read web pages using Excel VBA. How do I carry out this task? Is it even possible?

Answer 1

You can use VBA to automate IE (plenty of examples via Google) or you can fetch the page directly using an instance of MSHTTP (ditto plenty of examples on the web). Which is best for your needs will depend on exactly what you want to do. Hard to say more without more detailed requirements.

Answer 2

Coming from Excel 2003, yes this is possible - you may use the SHDocVw.InternetExplorer and MSHTML.HTMLDocument objects to call a web page, gain control over and interact with the DOM object. After creating references to Microsoft HTML Object Library (...\system32\MSHTML.TLB) and Microsoft Internet Control (...\system32\ieframe.dll) you can play with the following example:

Sub Test()
Dim Browser As SHDocVw.InternetExplorer
Dim HTMLDoc As MSHTML.HTMLDocument

    Set Browser = New SHDocVw.InternetExplorer                     ' create a browser
    Browser.Visible = True                                         ' make it visible
    Application.StatusBar = ".... opening page"
    Browser.navigate "http://www.google.com"                       ' navigate to page
    WaitForBrowser Browser, 10                                     ' wait for completion or timeout

    Application.StatusBar = "gaining control over DOM object"
    Set HTMLDoc = Browser.document                                 ' load the DOM object
    WaitForBrowser Browser, 10

    ' at this point you can start working with the DOM object. Usefull functions are
    ' With HTMLDoc
    '     .getElementByID(string)
    '     .getElementsByTagName(string)
    '     .getElementsByName(string)
    '     .getAttribute(string)
    '     .setAttribute string, string     .... to change field values, click a button etc.
    ' End With

    Application.StatusBar = ""                                      ' time to clean up
    Browser.Quit
    Set HTMLDoc = Nothing
    Set Browser = Nothing
End Sub

Sub WaitForBrowser(Browser As SHDocVw.InternetExplorer, Optional TimeOut As Single = 10)
Dim MyTime As Single

    MyTime = Timer
    Do While Browser.Busy Or (Timer <= MyTime + TimeOut)
        DoEvents
    Loop

    ' if browser still busy after timeout, give up
    If Browser.Busy Then
        MsgBox "I waited for " & Timer - MyTime & " seconds, but browser still busy" & vbCrLf & _
               "I give up now!"
        End
    End If
End Sub