R -RMySQL- how to save more sql queries to file?

问题

I do some data analysis in R. On end of script I want save my results to file. I know there is more options how to do it, but they don't work properly. When I try sink() it works but it give me :

<MySQLResult:1,5,1>
         host logname user                time                                                                       request_fline status
1 142.4.5.115      -    -  2018-01-03 12:08:58  GET /phpmyadmin?</script><script>alert('<!--VAIBS-->');</script><script> HTTP/1.1     400
                                                                size_varchar referer agent     ip_adress size_int cookie time_microsec filename
1  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0    445      -  142.4.5.115       445     -            159        -
  request_protocol keepalive request_method contents_of_foobar contents_of_notefoobar port child_id
<MySQLResult:1,5,1>
 [1] host                   logname                user                   time                   request_fline          status                
 [7] size_varchar           referer                agent                  ip_adress              size_int               cookie                     
<0 rows> (or 0-length row.names)

which is totally unusable because I cant export that type of data. If I try write.table it give file with one row which is possible read but after one row R skript and give me error : Error in isOpen(file, "w") : invalid connection and when I try write.csv result is same. And when I try lapply it give me just empty file.

There is my code :

fileConn<-file("outputX.txt")
fileCon2<-file("outputX.csv")
sink("outputQuery.txt")
for (i in 1:length(awq)){
  sql <- paste("SELECT * FROM mtable ORDER BY cookie LIMIT ", awq[i], ",1")
  nb <- dbGetQuery(mydb, sql)
  print (nb)
  write.table(nb, file = fileConn, append = TRUE, quote = FALSE, sep = " ", eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)
  write.csv(nb, file = fileCon2,row.names=FALSE, sep="  ")
  lapply(nb, write, fileConn, append=TRUE, ncolumns=7)
  writeLines(unlist(lapply(nb, paste, collapse=" ")))
}
sink()
close(fileConn)
close(fileCon2)

I am new in R, so I don't know what else should I try.What I want is 1 file where data will be print in form which is easy to read and export. For example tike this :

142.4.5.115  -   -  2018-01-03 12:08:58  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0
142.4.5.115  -   -  2018-01-03 12:10:23  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0
142.4.5.115  -   -  2018-01-03 12:12:41  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0
142.4.5.115  -   -  2018-01-03 12:15:29  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0

or this :

host,logname,user,time, request_fline status,size_varchar,referer agent,ip_adress,size_int,cookie,time_microsec,filename,request_protocol,keepalive,request_method,contents_of_foobar,contents_of_notefoobar port child_id
1 142.4.5.115  -   -  2018-01-03 12:08:58  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0
2 142.4.5.115  -   -  2018-01-03 12:10:23  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0
3 142.4.5.115  -   -  2018-01-03 12:12:41  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0
4 142.4.5.115  -   -  2018-01-03 12:15:29  GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1  400  Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0  445  -  142.4.5.115  445  -  145 -  HTTP/1.1  0  GET   -   -  80 7216  ?/><!--VAIBS-->   GET /phpmyadmin?/><!--VAIBS--> HTTP/1.1   -  0  /phpmyadmin   -  354 0

or something similar. Best of all, will be some help how to write write.table in loop without error. But I will welcome any functional solution. Best what I have is :

sql <- paste("SELECT * FROM idsaccess ORDER BY cookie LIMIT ", awq[1], ",1")
nb <- dbGetQuery(mydb, sql)
write.table(nb, file = fileConn, append = TRUE, quote = FALSE, sep = " ", eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)
fileConn<-file("outputX1.txt")
sql <- paste("SELECT * FROM idsaccess ORDER BY cookie LIMIT ", awq[2], ",1")
nb <- dbGetQuery(mydb, sql)
write.table(nb, file = fileConn, append = true, quote = FALSE, sep = " ", eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)

But this give every query to own file. And I don't want have every query in own file. Any help ?

回答1:

Simply concatenate all query dataframes into one large dataframe since they all share same structure, and then output to file in one call which is really the typical way to use write.table or its wrapper, write.csv:

Specifically, turn for loop:

for (i in 1:length(awq)){
  sql <- paste("SELECT * FROM mtable ORDER BY cookie LIMIT ", awq[i], ",1")
  nb <- dbGetQuery(mydb, sql)
}

Into lapply for a list of dataframes:

df_list <- lapply(1:length(awq), function(i) {
  sql <- paste0("SELECT * FROM mtable ORDER BY cookie LIMIT ", awq[i], ",1")
  nb <- dbGetQuery(mydb, sql)
})

Then, row bind with do.call to stack all dfs into a single dataframe and output to file:

final_df <- do.call(rbind, df_list)

write.table(final_df, file = "outputX.txt", append = true, quote = FALSE, sep = " ", 
            eol = "\n", na = "NA", row.names = FALSE, col.names = FALSE)

来源：https://stackoverflow.com/questions/49614053/r-rmysql-how-to-save-more-sql-queries-to-file

标签

rmysql