Scraping data from TripAdvisor using R

Question

I want to create a crawler that will scrape some data from Trip Advisor. Ideally, it will (a) identify the links to all locations to crawl, (b)

Answer 1

Basically, you can try to send a click event to the

. Something like this :

remDr$navigate(tu)
div <- remDr$findElement("class", "morePopularCities")
div$clickElement()

To expand all locations, you can possibly repeat the above logic in a while loop. Keep clicking on the

until no more items available (until the div no longer in the page) :

divs <- remDr$findElements("class", "morePopularCities")
while(length(divs )>0) {
  for(div in divs ){
    div$clickElement()
  }
  divs <- remDr$findElements("class", "morePopularCities")
}

I'm not fluent in R, you may find my code example not pretty, feel free to suggest.

0 讨论(0)

发布评论:

提交评论

---

•  加载中...