excel vba download text file from the internet that update every 5 minutes

Question

I wanted to download a file from this link: https://www.hko.gov.hk/tide/marine/data/ALL.txt

This file updates itself every 5 minutes. So I went on and create an excel VB

Answer 1

There may be caching. Try the following where a requestHeader is added to try and mitigate for potential caching. The other common alternative, adding a random number to the end of the URL doesn't seem to work for this site.

Option Explicit

Public Sub test()
    Dim i As Long
    For i = 1 To 3
        DownloadFile "https://www.hko.gov.hk/tide/marine/data/ALL.txt"
        Debug.Print Now
        Application.Wait Now + TimeSerial(0, 5, 0)
    Next
End Sub

Public Sub DownloadFile(ByVal link As String)
    Dim WinHttpReq As Object, oStream  As Object
    Set WinHttpReq = CreateObject("Microsoft.XMLHTTP")
    With WinHttpReq
        .Open "GET", link, False
        .setRequestHeader "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT"
        .send

        Debug.Print StrConv(.responsebody, vbUnicode)

        If WinHttpReq.Status = 200 Then
            Set oStream = CreateObject("ADODB.Stream")
            oStream.Open
            oStream.Type = 1
            oStream.Write .responsebody
            oStream.SaveToFile ThisWorkbook.Path & "\raw\" & "temp.csv", 2 ' 1 = no overwrite, 2 = overwrite
            oStream.Close
        End If

    End With

End Sub

Answer 2

QHarr's answer did not work for me. I tried many solutions, but it seems either they did not work with VBA or there was a need for an early binding call, which I did not like so I came up with the following method. Simply, we will try to delete the same url/file from the cache and then proceed into downloading the file:

'--- copy this to the top of the module (below Option Explicit) ---
'Declaration for deleting a file from the cache:
Private Declare PtrSafe Function DeleteUrlCacheEntry Lib "wininet.dll" Alias _
    "DeleteUrlCacheEntryA" (ByVal lpszUrlName As String) As Long

The function should be modified as follows:

Function DownloadFile(link As String)
    Dim WinHttpReq As Object
    
    'First delete the file from cache:
    'On Error Resume Next
    DeleteUrlCacheEntry url
    
    Set WinHttpReq = CreateObject("Microsoft.XMLHTTP")
    
    WinHttpReq.Open "GET", link, False, "username", "password"
    WinHttpReq.send
    
    myURL = WinHttpReq.responseBody
    If WinHttpReq.Status = 200 Then
        Set oStream = CreateObject("ADODB.Stream")
        oStream.Open
        oStream.Type = 1
        oStream.Write WinHttpReq.responseBody
        oStream.SaveToFile ThisWorkbook.Path & "\raw\" & "temp.csv", 2 ' 1 = no overwrite, 2 = overwrite
        oStream.Close
    End If

End Function