Multiple regression analysis in R using QR decomposition

Question

I am trying to write a function for solving multiple regression using QR decomposition. Input: y vector and X matrix; output: b, e, R^2. So far I`ve got this and am terribly

Answer 1

It all depends on how much of R's built-in facility you are allowed to use to solve this problem. I already know that lm is not allowed, so here is the rest of the story.

---

If you are allowed to use any other routines than lm

Then you can simply use lm.fit, .lm.fit or lsfit for QR-based ordinary least squares solving.

lm.fit(X, y)
.lm.fit(X, y)
lsfit(X, y, intercept = FALSE)

Among those, .lm.fit is the most light-weighed, while lm.fit and lsfit are pretty similar. The following is what we can do via .lm.fit:

f1 <- function (X, y) {
  z <- .lm.fit(X, y)
  RSS <- crossprod(z$residuals)[1]
  TSS <- crossprod(y - mean(y))[1]
  R2 <- 1 - RSS / TSS
  list(coefficients = z$coefficients, residuals = z$residuals, R2 = R2)
  }

In the question by your fellow classmate: Toy R function for solving ordinary least squares by singular value decomposition, I have already used this to check the correctness of SVD approach.

---

If you are not allowed to use R's built-in QR factorization routine qr.default

If .lm.fit is not allowed, but qr.default is, then it is also not that complicated.

f2 <- function (X, y) {
  ## QR factorization `X = QR`
  QR <- qr.default(X)
  ## After rotation of `X` and `y`, solve upper triangular system `Rb = Q'y` 
  b <- backsolve(QR$qr, qr.qty(QR, y))
  ## residuals
  e <- as.numeric(y - X %*% b)
  ## R-squared
  RSS <- crossprod(e)[1]
  TSS <- crossprod(y - mean(y))[1]
  R2 <- 1 - RSS / TSS
  ## multiple return
  list(coefficients = b, residuals = e, R2 = R2)
  }

If you further want variance-covariance of estimated coefficients, follow How to calculate variance of least squares estimator using QR decomposition in R?.

---

If you are not even allowed to use qr.default

Then we have to write QR decomposition ourselves. Writing a Householder QR factorization function in R code is giving this.

Using the function myqr there, we can write

f3 <- function (X, y) {
  ## our own QR factorization
  ## complete Q factor is not required
  QR <- myqr(X, complete = FALSE)
  Q <- QR$Q
  R <- QR$R
  ## rotation of `y`
  Qty <- as.numeric(crossprod(Q, y))
  ## solving upper triangular system
  b <- backsolve(R, Qty)
  ## residuals
  e <- as.numeric(y - X %*% b)
  ## R-squared
  RSS <- crossprod(e)[1]
  TSS <- crossprod(y - mean(y))[1]
  R2 <- 1 - RSS / TSS
  ## multiple return
  list(coefficients = b, residuals = e, R2 = R2)
  }

f3 is not extremely efficient, as we have formed Q explicitly, even though it is the thin-Q factor. In principle, we should rotate y along with the QR factorization of X, thus Q needs not be formed.

---

If you want to fix your existing code

This requires some debugging effort so would take some time. I will make another answer regarding this later.