Wait until page is loaded with Selenium WebDriver for Python

Question

I want to scrape all the data of a page implemented by a infinite scroll. The following python code works.

for i in range(100):
    driver.execute_script(\"w         


        

Answer 1

Have you tried driver.implicitly_wait. It is like a setting for the driver, so you only call it once in the session and it basically tells the driver to wait the given amount of time until each command can be executed.

driver = webdriver.Chrome()
driver.implicitly_wait(10)

So if you set a wait time of 10 seconds it will execute the command as soon as possible, waiting 10 seconds before it gives up. I've used this in similar scroll-down scenarios so I don't see why it wouldn't work in your case. Hope this is helpful.

To be able to fix this answer, I have to add new text. Be sure to use a lower case 'w' in implicitly_wait.