selenium python webscrape fails after first iteration

Question

Im iterating through tripadvisor to save comments(non-translated, original) and translated comments (from portuguese to english). So the scraper first selects portuguese com

Answer 1

There are 3 problems in your code

1. Inside method save_comments(), at the driver.find_element_by_class_name("ui_close_x").click().perform(), the method click() of a webelement is not an ActionChain so you cannot call perform(). Therefore, that line should be like this:

driver.find_element_by_class_name("ui_close_x").click()

2. Inside method save_comments(), at the com= driver.find_element_by_xpath(".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']"), you find the element when it doesn't appear yet. So you have to add wait before this line. Your code should be like this:

wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, ".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']")))
com= driver.find_element_by_xpath(".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']")

3. There are 2 buttons which can open the review, one is displayed and one is hidden. So you have to skip the hidden button.

if not i.is_displayed():
    continue
driver.execute_script("arguments[0].click()",i)