regarding the failure of stepwise variable selection in lm

问题

I built a regression model using all the variables at first.

full.model<-lm(y~as.matrix(x))

Then I tried to use step-wise variable selection

reduce.model<-step(full.model,direction="backward")

The running result is shown as follows, looks like it does not do anything. What is the problem of this scenario. I also include the detail of full.model in the following.

> reduce.model<-step(full.model,direction="backward")
   Start:  AIC=-121.19
   y ~ as.matrix(x)

                 Df   Sum of Sq        RSS     AIC
  <none>                               1.1 -121.19
   - as.matrix(x) 37     21550         21550.7  310.36

回答1:

You are using lm(...) incorrectly. In general, it is always better to build a model formula by referencing columns in a data frame. Try it this way:

# example data - you have this already...
set.seed(1)            # for reproducible example
x <- sample(1:500,500) # need this so predictors are not perfectly correlated.
x <- matrix(x,nc=5)    # 100 rows, 5 cols
y <- 1+ 3*x[,1]+2*x[,2]+4*x[,5]+rnorm(100)  # y depends on variables 1, 2, 5 only

# you start here...
df <- data.frame(y,as.matrix(x))
full.model <- lm(y ~ ., df)                 # include all predictors
step(full.model,direction="backward")
# Start:  AIC=3.32
# y ~ X1 + X2 + X3 + X4 + X5
# ...
#
# Step:  AIC=1.38
# y ~ X1 + X2 + X3 + X5
# ...
# 
# Step:  AIC=-0.53
# y ~ X1 + X2 + X5
# 
#        Df Sum of Sq    RSS    AIC
# <none>                  92  -0.53
# - X2    1     53912  54004 635.16
# - X1    1    110870 110961 707.18
# - X5    1    235260 235352 782.37
#
# Call:
# lm(formula = y ~ X1 + X2 + X5, data = df)
# 
# Coefficients:
# (Intercept)           X1           X2           X5  
#       1.367        2.998        2.006        3.997

来源：https://stackoverflow.com/questions/26825101/regarding-the-failure-of-stepwise-variable-selection-in-lm

标签

statistics

regression