How to accumulate the results of readr::read_lines_chunked?

戏子无情 提交于 2019-12-25 01:37:36

问题


I'm using readr::read_lines_chunked in the following way:

if(!require(readr)) install.packages("readr", repos = "http://cran.us.r-project.org")

mytb <- NULL
read_lines_chunked(file="/tmp/huge.xml", chunk_size=10, callback = function(xml, pos) {
   // extract values from xml into tmp
   if (is.null(mytb)) {
      users <- as_tibble(tmp)
   } else {
      users <- bind_rows(users, as_tibble(tmp)) 
   }
})

but this doesn't work as mytb always ends up being null ... how do you accumulate the results into a tibble?


回答1:


I found the solution. This package has a group of callback handlers that wrap the custom handler. So this is how it works:

mytb <- read_lines_chunked(file="/tmp/huge.xml", chunk_size=10, callback = DataFrameCallback$new(function(xml, pos) {
   // extract values from xml into tmp
   as_tibble(tmp)
}))

Note the DataFrameCallback$new(...) decorator and returning the tibble I want to stitch together as rbind.



来源:https://stackoverflow.com/questions/59314478/how-to-accumulate-the-results-of-readrread-lines-chunked

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!