Applying by function to lm()

Question

I am new to R and I am just learning about the apply functions and how they work. I simply want to extract the coefficients from a lm fit on a var

Answer 1

Try this instead:

tmp <- by(d, year, function(d.subset) lm(x~color, data=d.subset))

Answer 2

In addition to doing this with base R functions (which is a good idea to learn how they work) it's also worth looking at who may have worked on this before. Fitting linear models to each subset of a data set is something that comes up a bunch, and it turns out people have built some convenient tools specifically for this.

Try looking at the package lme4 and the function lmList. From the example in ?lmList,

(fm1 <- lmList(Reaction ~ Days | Subject, sleepstudy))

which fits the linear model Reaction ~ Days separately for each Subject. This is conveient because extractor methods work directly on fm1:

> coef(fm1)
    (Intercept)      Days
308    244.1927 21.764702
309    205.0549  2.261785
310    203.4842  6.114899
330    289.6851  3.008073
331    285.7390  5.266019
332    264.2516  9.566768
333    275.0191  9.142045
334    240.1629 12.253141
335    263.0347 -2.881034
337    290.1041 19.025974
349    215.1118 13.493933
350    225.8346 19.504017
351    261.1470  6.433498
352    276.3721 13.566549
369    254.9681 11.348109
370    210.4491 18.056151
371    253.6360  9.188445
372    267.0448 11.298073

(the row numbers are the id values for the Subjects) See the help file at ?lmList to see what other methods are available for things like confidence intervals, plotting, etc.

Answer 3

This is much easier in more modern packages, e.g. data.table:

library(data.table)
setDT(d)
d[ , .(reg = list(lm(x ~ color))), by = year]
#    year  reg
# 1: 2012 <lm>
# 2: 2006 <lm>
# 3: 2011 <lm>
# 4: 2008 <lm>
# 5: 2007 <lm>
# 6: 2010 <lm>
# 7: 2009 <lm>

The reg column has lm objects; note we need to wrap lm in list(.) so that data.table doesn't confuse the plain list (note that is.list(lm(x ~ color, data = d)) is TRUE.