Check if it's possible to scroll down with RSelenium

后端 未结 1 1453
臣服心动
臣服心动 2021-01-15 19:49

I\'m using RSelenium to automatically scroll down a social media website and save posts. Sometimes I get to the bottom of the webpage and no more posts can be loaded as no m

1条回答
  •  醉梦人生
    2021-01-15 20:35

    Stumbled across a way to do this in Python here and modified it to work in R. Below is a now-working update of the original code I posted above.

    # Open webpage
    library(RSelenium)
    rD = rsDriver(browser = "firefox")
    remDr = rD[["client"]]
    url = "https://stocktwits.com/symbol/NZDCHF"
    remDr$navigate(url) 
    
    # Keep scrolling down page, loading new content each time. 
    last_height = 0 #
    repeat {   
      remDr$executeScript("window.scrollTo(0,document.body.scrollHeight);")
      Sys.sleep(3) #delay by 3sec to give chance to load. 
    
      # Updated if statement which breaks if we can't scroll further 
      new_height = remDr$executeScript("return document.body.scrollHeight")
      if(unlist(last_height) == unlist(new_height)) {
        break
      } else {
        last_height = new_height
      }
    }
    

    0 讨论(0)
提交回复
热议问题