Planned contrasts using ezANOVA output in R

Question

I\'ve been looking into using planned contrasts as opposed to post-hoc t-tests. I typically use ezANOVA (Type III ANOVA) but it seems that conducting planned contra

Answer 1

The emmeans package provides the appropriate functionality to calculate custom contrasts/arbitrary linear functions of the estimates marginal means (EMMs) for aov and aovlist objects (see here for a full list of supported models).

In the following I use the ANT data set which comes with the ez package.

First we set up a mixed factorial ANOVA using ezANOVA. Note that one needs to set orthogonal contrasts in order to get meaningful Type-III tests (see, e.g., John Fox' answer here).

library("ez")
library("emmeans")

# set orthogonal contrasts
options(contrasts = c("contr.sum", "contr.poly"))

data(ANT)
rt_anova <- ezANOVA(data = ANT[ANT$error == 0, ], 
                    dv = rt,
                    wid = subnum, 
                    within = .(cue, flank),
                    between = group,
                    type = 3,
                    return_aov = TRUE)

We can then calculate the EMMs for, say all group-flank combinations.

emm <- emmeans(rt_anova$aov, ~ group * flank)
emm
## group     flank         emmean       SE    df lower.CL upper.CL
## Control   Neutral     381.5546 1.735392 53.97 378.0753 385.0339
## Treatment Neutral     379.9286 1.735392 53.97 376.4493 383.4079
## Control   Congruent   381.6363 1.735392 53.97 378.1570 385.1155
## Treatment Congruent   379.7520 1.735392 53.97 376.2727 383.2313
## Control   Incongruent 466.6770 1.735392 53.97 463.1977 470.1563
## Treatment Incongruent 452.2352 1.735392 53.97 448.7559 455.7145

Now it is easy to calculate all pairwise comparisons or any desired contrast on these EMMs.
See also this book chapter and my answer here if you need a bit more detail on how one can derive the contrasts weights from the hypotheses.

# all pairwise comparisons 
pairs(emm, adjust = "Holm")
## contrast                                        estimate       SE    df t.ratio p.value
## Control,Neutral - Treatment,Neutral           1.62594836 2.454215 53.97   0.663  1.0000
## Control,Neutral - Control,Congruent          -0.08167403 2.473955 36.00  -0.033  1.0000
## Control,Neutral - Treatment,Congruent         1.80259257 2.454215 53.97   0.734  1.0000
## Control,Neutral - Control,Incongruent       -85.12239797 2.473955 36.00 -34.407  <.0001
## Control,Neutral - Treatment,Incongruent     -70.68062093 2.454215 53.97 -28.800  <.0001
## Treatment,Neutral - Control,Congruent        -1.70762239 2.454215 53.97  -0.696  1.0000
## Treatment,Neutral - Treatment,Congruent       0.17664421 2.473955 36.00   0.071  1.0000
## Treatment,Neutral - Control,Incongruent     -86.74834633 2.454215 53.97 -35.347  <.0001
## Treatment,Neutral - Treatment,Incongruent   -72.30656929 2.473955 36.00 -29.227  <.0001
## Control,Congruent - Treatment,Congruent       1.88426660 2.454215 53.97   0.768  1.0000
## Control,Congruent - Control,Incongruent     -85.04072394 2.473955 36.00 -34.374  <.0001
## Control,Congruent - Treatment,Incongruent   -70.59894690 2.454215 53.97 -28.766  <.0001
## Treatment,Congruent - Control,Incongruent   -86.92499054 2.454215 53.97 -35.419  <.0001
## Treatment,Congruent - Treatment,Incongruent -72.48321351 2.473955 36.00 -29.299  <.0001
## Control,Incongruent - Treatment,Incongruent  14.44177704 2.454215 53.97   5.884  <.0001
## 
## Results are averaged over the levels of: cue 
## P value adjustment: holm method for 15 tests 

# custom contrasts
contrast(
  emm, 
  list(c1 = c(1, -1, 0, 0, 0, 0), # reproduces first pairwise comparison
       # emmean of row 1 - (emmean of row 1 + emmean of row 2) / 2; see EMMs table
       # 381.5546 - (379.9286 + 381.6363) / 2
       c2 = c(1, -0.5, -0.5, 0, 0, 0))
 )
 ## contrast  estimate       SE    df t.ratio p.value
 ## c1       1.6259484 2.454215 53.97   0.663  0.5105
 ## c2       0.7721372 2.136825 43.84   0.361  0.7196

The same applies for purely within-subjects ANOVAs or between-subjects ANOVAs.

# within-subjects ANOVA
rt_anova_wi <- ezANOVA(data = ANT[ANT$error == 0, ], 
                    dv = rt,
                    wid = subnum, 
                    within = .(cue, flank),
                    type = 3,
                    return_aov = TRUE)

emm <- emmeans(rt_anova_wi$aov, ~ cue * flank)
contrast(
  emm, 
  list(c1 = c(1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), 
       c2 = c(1, -0.5, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0))
) 
## contrast estimate       SE     df t.ratio p.value
## c1       47.31005 3.802857 170.34  12.441  <.0001
## c2       50.35320 3.293371 170.34  15.289  <.0001

# between-subjects ANOVA
rt_anova_bw <- ezANOVA(data = ANT[ANT$error == 0, ], 
                       dv = rt,
                       wid = subnum, 
                       within_full = .(cue, flank), 
                       between = group,
                       type = 3,
                       return_aov = TRUE)

emm_bw <- emmeans(rt_anova_bw$aov, ~ group)
# custom linear function
contrast(
  emm_bw, 
  list(c1 = c(2/3, 1/2)) 
)
## contrast estimate        SE df t.ratio p.value
## c1       475.2899 0.8213448 18 578.673  <.0001