cbind() is changing date formatting

Question

I\'m trying to create a subset of a data frame and when I do so, R switches the formatting of the date column. Any idea why or how to fix this?

> head(spyPr2         


        

Answer 1

Obvious answer is don't do subsetting like that! Use the appropriate tools. What is wrong with

spyPr2.new <- spyPr2[, c("Date", "Close", "Adj.Close")]

?

To explain the behaviour you are seeing, you need to understand what $ returns and how cbind() works. cbind() is one of those oddities in R wherein method dispatch is not done via the usual method but is instead handled via special code buried in the internals of R. This is all the R code behind cbind():

> cbind
function (..., deparse.level = 1) 
.Internal(cbind(deparse.level, ...))
<bytecode: 0x24fa0c0>
<environment: namespace:base>

Not much help, eh? There are methods for data frames and "ts" objects however:

> methods(cbind)
[1] cbind.data.frame cbind.ts*       

   Non-visible functions are asterisked

Before I do the reveal, also note what $ returns (dat2 is your 6 lines of data after converting Date to a "Date" object):

> str(dat2$Date)
 Date[1:6], format: "2011-12-30" "2011-12-29" "2011-12-28" "2011-12-27" ...

This is a "Date" object, which is a special vector really.

> class(dat2$Date)
[1] "Date"

The key thing is that it is not a data frame. So when you use cbind(), the internal code is seeing three vectors and the internal code creates a matrix.

> (c1 <- cbind(dat2$Date, dat2$Close, dat2$Adj.Close))
      [,1]   [,2]   [,3]
[1,] 15338 125.50 125.50
[2,] 15337 126.12 126.12
[3,] 15336 124.83 124.83
[4,] 15335 126.49 126.49
[5,] 15331 126.39 126.39
[6,] 15330 125.27 125.27
> class(c1)
[1] "matrix"

There can only be numeric or character matrices in R so the Date object is converted to a numeric vector:

> as.numeric(dat2$Date)
[1] 15338 15337 15336 15335 15331 15330

to allow cbind() to produce a numeric matrix.

You can force the use of the data frame method by calling it explicitly and it does know how to handle "Date" objects and so doesn't do any conversion:

> cbind.data.frame(dat2$Date, dat2$Close, dat2$Adj.Close)
   dat2$Date dat2$Close dat2$Adj.Close
1 2011-12-30     125.50         125.50
2 2011-12-29     126.12         126.12
3 2011-12-28     124.83         124.83
4 2011-12-27     126.49         126.49
5 2011-12-23     126.39         126.39
6 2011-12-22     125.27         125.27

However, all the explanation aside, you are trying to do the subsetting in a very complex manner. [ as a subset function works just fine:

> dat2[, c("Date", "Close", "Adj.Close")]
        Date  Close Adj.Close
1 2011-12-30 125.50    125.50
2 2011-12-29 126.12    126.12
3 2011-12-28 124.83    124.83
4 2011-12-27 126.49    126.49
5 2011-12-23 126.39    126.39
6 2011-12-22 125.27    125.27

subset() is also an option but not needed here:

> subset(dat2, select = c("Date", "Close", "Adj.Close"))
        Date  Close Adj.Close
1 2011-12-30 125.50    125.50
2 2011-12-29 126.12    126.12
3 2011-12-28 124.83    124.83
4 2011-12-27 126.49    126.49
5 2011-12-23 126.39    126.39
6 2011-12-22 125.27    125.27

Answer 2

I think I might call this a hidden instance of the drop = FALSE gotcha with data frames.

When you use cbind, it only uses the data frame method if at least one of the components are also data frames. Otherwise, everything is converted to a single type in order to construct a matrix.

Thus, calling cbind on elements like spyPr2$Date or spyPr2[,'Date'] will result in a matrix (losing the date structure), which will not be magically restored by wrapping it all in data.frame.

You can do this if you use [ to select each column, but only by using drop = FALSE which prevents R from converting the result to a vector (which lands you right back where you started with R coercing the result to a matrix):

cbind(spyPr2[,'Date',drop = FALSE],spyPr2[,'Close'])

is sufficient, since you only need one of the components to be a data frame.

But Gavin is right in general, you shouldn't be subsetting your data frame this way.