To pause code until a web page is fully loaded, I\'ve been using the method below with great success almost all of the time.
Do While objIE.Busy = True Or ob
Functionality of webpages is very different, so there is no solution that will fit to all of them.
Regarding your example, your workaround is a working solution, the code might be like:
Sub TestIE()
Dim q
With CreateObject("InternetExplorer.Application")
.Visible = True
.Navigate "https://www.homedepot.ca/en/home/p.dry-cloth-refills-32---count.1000660019.html"
' Wait IE
Do While .readyState < 4 Or .Busy
DoEvents
Loop
' Wait document
Do While .document.readyState <> "complete"
DoEvents
Loop
' Wait element
Do
q = .document.querySelector(".product-total-price").innerText
If Left(q, 1) <> "-" Then Exit Do
DoEvents
Loop
.Quit
End With
Debug.Print q
End Sub
Anyway, you need to look into the webpage loading process, XHRs and DOM modifications, using browser developer tools (F12). Going that way, you may find that one of the numerous XHRs returns the price in JSON format. It's logged on network tab of browser developer tools right before the price appearing while the page is loading. That XHR is made by one of the loaded JS, notably after the page loaded event. Try this URL (I just copied it from network tab):
https://www.homedepot.ca/homedepotcacommercewebservices/v2/homedepotca/products/1000660019/localized/9999?catalogVersion=Online&lang=en
So you may just reproduce that XHR and extract the price by splitting:
Sub TestXHR()
Dim q
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "https://www.homedepot.ca/homedepotcacommercewebservices/v2/homedepotca/products/1000660019/localized/9999?catalogVersion=Online&lang=en", False
.Send
q = .ResponseText
End With
q = Replace(q, " : ", ":")
q = Split(q, """displayPrice""", 2)(1)
q = Split(q, """formattedValue"":""", 2)(1)
q = Split(q, """", 2)(0)
Debug.Print q
End Sub
But again, there is no common case.
You may also use JSON parser, take a look at some examples.