How to click on Load More button within Google Trends and print all the titles through Selenium and Python
this time I would like to click a button in order to load more real-time searches. Here is the link of the website:
https://trends.google.com/trends/trendingsearches
-
To click on
LOAD MOREbutton to load more real-time searches and then to print them you can use the following solution:Code Block:
# -*- coding: UTF-8 -*- from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.common.exceptions import TimeoutException options = webdriver.ChromeOptions() options.add_argument("start-maximized") options.add_argument('disable-infobars') driver=webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe') driver.get("https://trends.google.com/trends/trendingsearches/realtime?geo=AR&category=all") myLength = len(WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='title']")))) while True: driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") try: WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='feed-load-more-button'][@ng-click=\"ctrl.loadMoreFeedItems()\"]"))).click() WebDriverWait(driver, 20).until(lambda driver: len(driver.find_elements_by_xpath("//div[@class='title']")) > myLength) titles = driver.find_elements_by_xpath("//div[@class='title']") myLength = len(titles) except TimeoutException: break for title in titles: print(title.text) driver.quit()Console Output:
Nikola Kalinic • 2018 World Cup • Zlatko Dalic Vicentico • Valeria Bertuccelli • Florián Fernandez Capello Mauricio Macri • Juan José Aranguren • Argentina • Jorge Lanata Goalkeeper • 2018 World Cup • Mohamed El-Shenawy • Ahmed El-Shenawy Argentina • Manuel Belgrano • María Josefa Ezcurra • Mercedes Tomasa de San Martín y Escalada South Korea national football team • Russia • Zlatan Ibrahimovic Italian nationality law • Mar del Plata Hirving Lozano • 2018 World Cup • Earthquake Central Bank of Argentina • Luis Caputo • Federico Sturzenegger • Nicolás Dujovne Porsche Mission E • Electric vehicle • Car • Electric car Intelligence • Intelligence quotient • Research Call for bids • National Flag Memorial • Monument • Rosario • Pablo Javkin Journalist • Club Atlético Belgrano • Córdoba • Manuel Belgrano France • Spain • Immigration Natti Natasha • Zum Zum • Daddy Yankee • R.K.M & Ken-Y • Francisco José Arcángel Ramos Mercosur • Pacific Alliance • Pact • Paraguay World Cup • England national football team • Romelu Lukaku • Dries Mertens House arrest • Detention • Robbery Argentina • Emerging markets • MSCI • Market • Finance • Morgan Stanley Eurozone • Budget • France • Bruno Le Maire • Germany • Finance Intelligence • Intelligence quotient • Research Brazil national football team • Tite • Philippe Coutinho • Rostov Arena 2018 World Cup • Aleksandar Kolarov • Guillermo Ochoa 2018 World Cup • Spain Sloth • Giant ground sloth • Fossil • San Pedro Government • Inflation • Alfredo Cornejo • Mendoza • Ministry of Finance Corrientes • Cannabis • Tractor • Argentine Naval Prefecture Argentine Chamber of Deputies • Chubut Province • Peronism • Front for Victory Jorge Rial • Father's Day • Intrusos en el espectáculo Debt • Debt relief • Refinancing • Chaco Province Polygraph Malaria • Paraguay • Americas General Confederation of Labour • Unemployment • Trade union Earthquake • Japan Angela Merkel • Europe • Germany • Emmanuel Macron • European Union Colombia • Ivan Duque • Gustavo Petro Lynx • Wildcat • Cat Rafael Márquez • 2018 World Cup • Antonio Carbajal Superior Council for Private Enterprise • Nicaraguan Social Security Institute 2018 World Cup • William Kvist • Jefferson Farfán La Plata • Controlled-access highway • Buenos Aires Province • Ensenada Leonardo Mayer • Kevin Anderson • ATP World Tour 500 series • Association of Tennis Professionals Cristiano Ronaldo • 2018 World Cup • Spain Casino • Mendoza • Hyatt Mauricio Macri • Jorge Lanata • Argentina • Alejandro Wiebe Drug Buenos Aires Province • Judiciary 2018 World Cup • Lionel Messi • Russia • Diego Maradona • Mario Kempes Santiago • Natural environment • Air pollution Chimbas Department Chaco Province • Judiciary • Unemployment Hailey Rhode Baldwin • Justin Bieber 2018 World Cup • Russia Mauricio Macri • Luis Caputo • Politician • Argentine Chamber of Deputies Trade war • Agriculture • Donald Trump • China 2018 World Cup • Dennis te Kloese • Juan Carlos Osorio Aaron Ramsey • 2018 World Cup • Arsenal F.C. • Football player • Gareth Bale 2018 World Cup • Aleksandar Kolarov Iñaki Urdangarin • Cristina Federica, Infanta of Spain • Spain • Luis Roldán Argentina • Electric car San Lorenzo de Almagro • Liga Nacional de Básquet • Corrientes Christina Aguilera • Jimmy Fallon • Liberation • New York City Intelligence quotient • Intelligence • Flynn effect • Research María Eugenia Vidal • Republican Proposal • Horacio Rodríguez Larreta Natural environment • Pollution • Plastic pollution Funding • International Finance Corporation • World Bank • Córdoba Face • Zygomatic bone • Rosario Cristóbal López • Administración Federal de Ingresos Públicos • Ricardo Echegaray Club Atlético Belgrano • Lucas Bernardi • Superliga Argentina de Fútbol Juan José Aranguren • Mauricio Macri • Energy • YPF Greater Buenos Aires • Motor coach Stranger Things • MTV Movie & TV Awards Peru • Lima • Earthquake • 2018 World Cup • Geophysics Institute of Peru Juan Carlos Osorio • 2018 World Cup 2018 World Cup • Player • Russia • Lionel Messi Raffaella Carrà America's Got Talent • Janis Joplin • Howie Mandel Homicide • Detention La Plata • Julio Garro • Víctor Manuel Fernández • María Eugenia Vidal Mariano Arcioni • Chubut Province • Payment Angela Merkel • Horst Seehofer • Germany • Government • Human migration Israel • Benjamin Netanyahu • Syria • Iran • Ali Khamenei • Vladimir Putin Yerba mate • Cannabis • Lomas de Zamora • Detention • Mate Senate • Argentine Chamber of Deputies 2010 FIFA World Cup • South Africa Handball • Chile • Argentina national football team 2018 World Cup • Russia • Terrorism • Attack Abortion • Conscientious objector Rosario • Trade Natalie Weber • Mauro Zárate • Pampita Water • Cipolletti • Pressure • Cleaning Chimbas NATO • Military exercise • Russia Federico Sturzenegger • Economist • Argentina • Arnaldo Bocco • Martín Redrado Nolle prosequi 2018 World Cup • Russia • Paolo Guerrero • Christian Cueva Ricardo Darín • Valeria Bertuccelli • Érica Rivas Joachim Löw • 2018 World Cup Israel • Golan Heights • Syria • Donald Trump Meningitis • Bacteria • Salta • Streptococcus pneumoniae ANSES • Subsidy Argentine Chamber of Deputies • Radical Civic Union • Cambiemos • Martín Demichelis • 2018 World Cup • Russia Santiago de Compostela • Pilgrim • Galicia • Spain Refugee • Pope Francis • Human migration Stranger Things • Father's Day • Joe Keery Ivan Duque • President of Colombia • Latin America General Confederation of Labour • Unemployment • General strike • Buenos Aires Province Juan Carlos Osorio • Faustino Asprilla • 2018 World Cup Germán Burgos • Club Atlético River Plate • Diego Simeone • Atlético Madrid Nissan Navara • Pickup truck • Automotive industry Baobab • Research • Tree Susana Giménez • Alejandro Wiebe • Argentina • Telefe Wanda Nara • Maxi López • Mauro Icardi Damir Skomina • 2018 World Cup • Colombia • Referee • Russia • Mehdi Abid Charef Diabetes mellitus • Visual perception • Diabetic retinopathy Berisso • Threat • La Plata • Search and seizure • School Game of Thrones • HBO • San Diego Comic-Con • Spin-off • George R. R. Martin Argentina • Duet • Traveling Wilburys Blood donation • Uruguay • Maldonado 2018 World Cup Group F • Mexico • Hirving Lozano Colombian presidential election, 2018 • Colombia • Juan Manuel Santos Colombia national football team Desertification • United Nations Convention to Combat Desertification Saski Baskonia • Liga ACB • Pablo Laso • Real Madrid C.F. Rosario de la Frontera • Salta • Spa town Neymar • 2018 World Cup • Philippe Coutinho • Tite • Russia Pocito Department Argentine peso • Depreciation • Central Bank of Argentina San Salvador de Jujuy • Buenos Aires International Book Fair • Fair • Córdoba • Shock Iceland • Immigration • Icelanders Mirtha Legrand Traffic collision • Wound 2018 World Cup • Russia • Vikings Mauricio Macri • Businessperson • Argentina • Economic development Ricardo Darín • Valeria Bertuccelli • Érica Rivas • Vicentico María Eugenia Vidal • Martiniano Molina • Quilmes Argentina women's national field hockey team • Julieta Jankunas • Argentina Prince Harry • Catherine, Duchess of Cambridge • British royal family Sergej Milinkovic-Savic • SS Lazio • 2018 World Cup Organ donation • Organ • Organ transplantation Sex education • San Fernando del Valle de Catamarca • Argentine Chamber of Deputies Margarita Stolbizer • Sergio Massa • Peronism • Justicialist Party • Elisa Carrió Google Maps • Waze • Information • Radar Locomotive • Japan Spain • Human migration • Immigration • France • Valencia • Carmen Calvo Poyato Joaquín Sabina • Madrid Jorge Lorenzo • Marc Márquez • MotoGP • Dani Pedrosa Manuel Belgrano 2018 World Cup • Russia • Mohamed Salah • Denis Cheryshev María Eugenia Vidal • Teacher Remand Magistrate • Judiciary • Research Laura Bush • Donald Trump • Immigration • George W. Bush • Melania Trump Sebastián Piñera • Chile • Michelle Bachelet School • Gender identity • Discrimination • National Institute Against Discrimination, Xenophobia and Racism Engineering • San Miguel de Tucumán School • Buenos Aires Province • Tariff Edgardo Bauza • Rosario Central • Marco Ruben • Rosario Bicameralism Russia • 2018 World Cup • Lionel Messi Rawson Department, San Juan El Litoral • Corrientes Province Abortion • PH: Podemos hablar • Andy Kusnetzoff • Charlotte Caniggia The Shining • Ewan McGregor • Doctor Sleep • Danny Torrance • Stephen King Paulo Ferrari • Rosario Central • Rosario • Superliga Argentina de Fútbol Carbon monoxide Ground frost • Fog • Rain and snow mixed • Cold • Posadas Argentine rock • Russia • Luis Alberto Spinetta • Gustavo Cerati • Charly García Incredibles 2 • Brad Bird • Pixar Carbohydrate • Dieting • Weight loss Uber • Mendoza Province • Government • System • Statute Paragliding Jorge Sampaoli • Pedro Pasculli • 2018 World Cup • Paulo Dybala • Russia Posadas • Cold Luis Miguel • Mexico Unidentified flying object • Russia • Phenomenon Lisandro Magallán • Boca Juniors • AFC Ajax • Wílmar Barrios Tandil • Fossil • Glyptodon Harry Kane • 2018 World Cup • Gareth Southgate Light welterweight • Boxing • Almirante Brown Partido • Mariano Cascallares New Jersey Small and medium-sized enterprises • Argentina • CAME - Argentina Confederation of Medium Enterprises Season • Ryan Murphy • American Horror Story: Murder House • Sarah Paulson Conflagration • Posadas Robot • Old age • China 2018 World Cup • Russia • Terrorism • Islamic State of Iraq and the Levant Christian Cueva • 2018 World Cup Argentina national football team • Volleyball • Argentina National Electoral Institute • Candidate • National Action Party • Mexico Horoscope • Astrological sign Mauricio Macri • María Eugenia Vidal • Ensenada • Ambulance • Cambiemos Trade • Tax deduction • Debt • Macroeconomics Cristiano Ronaldo • Pelé • 2018 World Cup • Miroslav Klose • Uwe Seeler Martín Miguel de Güemes • Juan Manuel Urtubey • Salta Province • Argentina Compressed natural gas • La Pampa Province • General Pico • Camuzzi Gas Pampeana Santa Fe • Provincial Hospital Dr. José María Cullen • Baleada National University of La Plata • Vocational school • School • Vocational education
讨论(0) -
You can use this:
wait = WebDriverWait(webdriver, 10) load_button = wait.until(EC.presence_of_element_located((By.XPATH, "//div[@class = 'feed-load-more-button']"))) load_button.click()讨论(0) -
for scrolling down you can use this code :
driver.execute_script("window.scrollTo(0, Y)")where Y : is the vertical position you want to scroll to.(you can provide the value of Y as 100,200,300 as per your requirement).
Once scroll down is done, you can use this code :
load_more_button = WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH, "//div[@ng-click='ctrl.loadMoreFeedItems()']"))) load_more_button.click()讨论(0) -
Find it with
find_elements_by_css_selectorand click on it. I don't think if you need to wait or scroll.element = driver.find_elements_by_css_selector("div[ng-click=\"ctrl.loadMoreFeedItems()\"]") element.click()
Read this answer, if you wonder why I have not used
xPath.讨论(0)
加载中...
- 热议问题