how to convert a very large dataset to xts? - as.xts fails on 1.5M rows

问题

I have the data:

dput(head(data))

> dput(head(data))
structure(list(Gmt.time = c("01.06.2015 00:00", "01.06.2015 00:01", 
"01.06.2015 00:02", "01.06.2015 00:03", "01.06.2015 00:04", "01.06.2015 00:05"
), Open = c(0.88312, 0.88337, 0.88377, 0.88412, 0.88393, 0.8838
), High = c(0.88337, 0.88378, 0.88418, 0.88418, 0.88393, 0.88393
), Low = c(0.883, 0.88337, 0.88374, 0.88394, 0.88368, 0.88362
), Close = c(0.88337, 0.88375, 0.88412, 0.88394, 0.8838, 0.88393
), Volume = c(83.27, 100.14, 117.18, 52.53, 77.69, 91.63)), .Names = c("Gmt.time", 
"Open", "High", "Low", "Close", "Volume"), row.names = c(NA, 
6L), class = "data.frame")
>

and no NA values

any(is.na(head(data)))
[1] FALSE

if i run this on the first few elements as in the data provided:

data_xts <- xts(head(data[,2:6]), as.POSIXct(head(data[,1]), format='%d.%m.%Y %H:%M'))

it works fine

but if i run on full dataset

> nrow(data)
[1] 1581120

i get:

> data_xts <- xts(data[,2:6], as.POSIXct(data[,1], format='%d.%m.%Y %H:%M'))
Error in xts(data[, 2:6], as.POSIXct(data[, 1], format = "%d.%m.%Y %H:%M")) : 
  'order.by' cannot contain 'NA', 'NaN', or 'Inf'

回答1:

If your timestamps are in GMT as column name implies, then as.POSIXct(data[,1], format='%d.%m.%Y %H:%M') may be returning NA because timezone has not been set to UTC and local timezone is assumed by default. You may have a timestamp that doesn't exist in local timezone, which would return NA. I.e., try as.POSIXct(data[,1], format='%d.%m.%Y %H:%M', tz = "GMT").

I am guessing that the first record returning NA contains a timestamp during an hour that is skipped due to daylight savings changes (i.e., does not exist) in your local time zone; as described here.

来源：https://stackoverflow.com/questions/50787374/how-to-convert-a-very-large-dataset-to-xts-as-xts-fails-on-1-5m-rows

标签

timestamp

time-series

xts

posixct