Waiting for a table to load completely using selenium with python

Question

I want to scrape some data from a page which is in a table. So I am only bothered about the data in the table. Earlier I was using Mechanize, but I found sometimes some of t

Answer 1

Perhaps you could use Selenium's expected conditions (http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp), e.g.

>>> from selenium import webdriver
>>> from selenium.webdriver.common.by import By
>>> from selenium.webdriver.support.ui import WebDriverWait
>>> from selenium.webdriver.support import expected_conditions as EC 
>>> 
>>> ff = webdriver.Firefox()
>>> ff.get("http://www.datatables.net/examples/data_sources/js_array.html")
>>> try:
...     element = WebDriverWait(ff, 10).until(EC.presence_of_element_located((By.ID, "example")))
...     print element.text
... finally:
...     ff.quit()
... 

Engine Browser Platform Version Grade
Gecko Firefox 1.0 Win 98+ / OSX.2+ 1.7 A
Gecko Firefox 1.5 Win 98+ / OSX.2+ 1.8 A
Gecko Firefox 2.0 Win 98+ / OSX.2+ 1.8 A
Gecko Firefox 3.0 Win 2k+ / OSX.3+ 1.9 A
Gecko Camino 1.0 OSX.2+ 1.8 A
Gecko Camino 1.5 OSX.3+ 1.8 A
Gecko Netscape 7.2 Win 95+ / Mac OS 8.6-9.2 1.7 A
Gecko Netscape Browser 8 Win 98SE+ 1.7 A
Gecko Netscape Navigator 9 Win 98+ / OSX.2+ 1.8 A
Gecko Mozilla 1.0 Win 95+ / OSX.1+ 1 A