Rolling regression with expanding window in R

问题

I would like to do a rolling linear regression, with expanding window, between two variables in a data frame, grouped by a third categorical column.

For example, in the toy data frame below, I would like to extract coefficient of lm(y~x) grouped by z using all rows until the row of interest. Thus for row 2, data set for regression will be rows 1:2, for row 3 will be rows 1:3, for row 4 will be just row 4 as it is the first row with categorical variable z= b

dframe<-data.frame(x=c(1:10),y=c(8:17), z=c("a","a","a","b","b","b","b","b","b","b"))

Using rollify function, I am able to get what I want except the expanding window. Below I have used a window size of 2

rol <- rollify(~coef(lm(.x~0+.y)),2) 
output<-dframe %>%  group_by(z) %>% mutate(tt=rol(x,y))

Specifically I do not know, how I can supply a variable window size to the rollify function. Is it possible?

Thinking broadly, what is an efficient way to do this operation? I need to do this on several 10000's of rows

回答1:

1) rollapplyr First split dframe and then run rollapplyr over each component of the split. Note that rollapplyr can take a vector of widths as the second argument.

library(zoo)

roll <- function(data, n = nrow(data)) {
  rollapplyr(1:n, 1:n, function(ix) coef(lm(y ~ x+0, data, subset = ix))[[1]])
}

L <- split(dframe[-3], dframe[[3]])
transform(dframe, roll = unlist(lapply(L, roll)))

giving:

    x  y z     roll
a1  1  8 a 8.000000
a2  2  9 a 5.200000
a3  3 10 a 4.000000
b1  4 11 b 2.750000
b2  5 12 b 2.536585
b3  6 13 b 2.363636
b4  7 14 b 2.222222
b5  8 15 b 2.105263
b6  9 16 b 2.007380
b7 10 17 b 1.924528

1a) A variation would be to use ave instead of split.

n <- nrow(dframe)
transform(dframe, roll = ave(1:n, z, FUN = function(ix) roll(dframe[ix, ]))

1b) This alternative has been added some time after the question was originally answered.

reg <- function(x) coef(lm(x[, 2] ~ x[, 1] + 0))
n <- nrow(dframe)
w <- ave(1:n, dframe$z, FUN = seq_along)
transform(dframe, 
  roll = rollapplyr(zoo(cbind(x, y)), w, reg, by.column = FALSE, coredata = FALSE))

2) dplyr/rollapplyr This is the same except we use dplyr to do the grouping. roll is from (1).

library(dplyr)
library(zoo)

dframe %>%
  group_by(z) %>%
  mutate(roll = roll(data.frame(x, y))) %>%
  ungroup

giving:

# A tibble: 10 x 4
# Groups:   z [2]
       x     y z      roll
   <int> <int> <fct> <dbl>
 1     1     8 a      8   
 2     2     9 a      5.20
 3     3    10 a      4.00
 4     4    11 b      2.75
 5     5    12 b      2.54
 6     6    13 b      2.36
 7     7    14 b      2.22
 8     8    15 b      2.11
 9     9    16 b      2.01
10    10    17 b      1.92

3) Base R This could also be done without any packages like this where L is from (1). The result is similar to (1).

transform(dframe, roll = unlist(lapply(L, function(data, n = nrow(data)) {
  sapply(1:n, function(i) coef(lm(y ~ x + 0, data, subset = 1:i))[[1]])
})))

3a) roll in (1) can be replaced with roll2 in the following which uses no packages and does not even use lm giving us another base R solution. Again, L is from (1).

roll2 <- function(data) with(data, cumsum(x * y) / cumsum(x * x))
transform(dframe, roll = unlist(lapply(L, roll2)))

回答2:

Here is an approach that will do the rolling traversal of the data frame you are asking about:

sapply(2:nrow(dframe), function(crt.row) {
    df = dframe[1:crt.row,]
    ## compute the statistics of interest on df (e.g. run the linear model),
    ## which is the subset of the original data frame that consists of rows 1 to
    ## current
    ##
    ## for example mean of x+y
    c(crt.row=crt.row, mystat=mean(df$x + df$y))
})

        [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
crt.row    2    3    4    5    6    7    8    9   10
mystat    10   11   12   13   14   15   16   17   18

来源：https://stackoverflow.com/questions/59461067/rolling-regression-with-expanding-window-in-r

标签

regression

rolling-computation