Write tweets from rtweets package to csv

问题

I'm unable to write tweets from search_tweet() in 'rtweet' package to csv. It throws the following error:

Here's a link to the question I previously asked, that has details on the type of search_tweet() object creates: Class and type of object is different in R. How should I make it consistent?

How should I write this files as csv?

library(rtweet)
comments <- search_tweets(
    queryString, include_rts = FALSE,
    n = 18000, type = "recent",
    retryonratelimit = FALSE)

write_csv(comments, "comments.csv", append =  TRUE)

Error: Error in stream_delim_(df, path, ..., bom = bom, quote_escape = quote_escape) : Don't know how to handle vector of type list.

class(comments)

"tbl_df" "tbl" "data.frame"

screen grab of comments

回答1:

The rtweet package has a function to export to CSV called write_as_csv but for some reason does not expose the append= option. You can take the code of that function and change it to add an append option. For example

write_as_csv2 <- function(x, file_name,
                         prepend_ids = TRUE,
                         na = "",
                         fileEncoding = "UTF-8", append=FALSE) {
  ## to minimize rounding
  op <- options()
  on.exit(options(op))
  options(scipen = 14, digits = 22)

  ## validate inputs
  stopifnot(is.data.frame(x), is.character(file_name), length(file_name) == 1L)
  if (!grepl("\\.csv$", file_name)) {
    file_name <- paste0(file_name, ".csv")
  }
  ## flatten data
  x <- flatten(x)
  if (prepend_ids) {
    x <- prepend_ids(x)
  }
  utils::write.table(x, file_name, row.names = FALSE, na = na,
    fileEncoding = fileEncoding, append=append, sep=",", dec=".", qmethod="double")

  # or
  # readr::write_csv(x, file_name, append =  append)
}
environment(write_as_csv2) <- asNamespace("rtweet")

Then you can call it like

write_as_csv2(comments, "comments.csv", append =  TRUE)

来源：https://stackoverflow.com/questions/59775074/write-tweets-from-rtweets-package-to-csv

标签

csv

dataframe

web-scraping

rtweet