Avoid Rate limit with rtweet get_timeline()

Question

Is there anyway to stop my loop from being interrupted by the rate limit? I would like my code to wait to execute until the time limit has passed if possible.

A side

Answer 1

I was able to resolve it by wrapping get_timeline() function in the following code. The function get_timeline_unlimited calls itself recursively after waiting the required time for the rate limit to reset. So far it worked well for me with no issues.

 get_timeline_unlimited <- function(users, n){

  if (length(users) ==0){
    return(NULL)
  }

  rl <- rate_limit(query = "get_timeline")

  if (length(users) <= rl$remaining){
    print(glue("Getting data for {length(users)} users"))
    tweets <- get_timeline(users, n, check = FALSE)  
  }else{

    if (rl$remaining > 0){
      users_first <- users[1:rl$remaining]
      users_rest <- users[-(1:rl$remaining)]
      print(glue("Getting data for {length(users_first)} users"))
      tweets_first <- get_timeline(users_first, n, check = FALSE)
      rl <- rate_limit(query = "get_timeline")
    }else{
      tweets_first <- NULL
      users_rest <- users
    }
    wait <- rl$reset + 0.1
    print(glue("Waiting for {round(wait,2)} minutes"))
    Sys.sleep(wait * 60)

    tweets_rest <- get_timeline_unlimited(users_rest, n)  
    tweets <- bind_rows(tweets_first, tweets_rest)
  }
  return(tweets)
}