Discrepancy emmeans in R (using ezAnova) vs estimated marginal means in SPSS

Kerwin Olfers Source

So this is a bit of a hail mary, but I'm hoping someone here has encountered this before. I recently switched from SPSS to R, and I'm now trying to do a mixed-model ANOVA. Since I'm not confident in my R skills yet, I use the exact same dataset in SPSS to compare my results.

I have a dataset with

dv = RT

within = Session (2 levels), Cue (3 levels), Flanker (2 levels)

between = Group(3 levels).

no covariates.

unequal number of participants per group level (25,25,23)

In R I'm using the ezAnova package to do the mixed-model anova:

results <- ezANOVA(
    data = ant_rt_correct
    , wid = subject
    , dv = rt
    , between = group
    , within = .(session, cue, flanker) 
    , detailed = T
    , type = 3
    , return_aov = T

In SPSS I use the following GLM:

GLM rt.1.center.congruent rt.1.center.incongruent rt.1.no.congruent rt.1.no.incongruent 
rt.1.spatial.congruent rt.1.spatial.incongruent rt.2.center.congruent rt.2.center.incongruent 
rt.2.no.congruent rt.2.no.incongruent rt.2.spatial.congruent rt.2.spatial.incongruent BY group 
/WSFACTOR=session 2 Polynomial cue 3 Polynomial flanker 2 Polynomial 
/WSDESIGN=session cue flanker session*cue session*flanker cue*flanker session*cue*flanker 

The results of which line up great, ie:

R: Session F(1,70) = 46.123 p = .000

SPSS: Session F(1,70) = 46.123 p = .000

I also ask for the means per cell using:

descMeans <- ezStats(
    data = ant_rt_correct
    , wid = subject
    , dv = rt
    , between = group
    , within = .(session, cue, flanker) #,cue,flanker)
    , within_full = .(location,direction)
    , type = 3

Which again line up perfectly with the descriptives from SPSS, e.g. for the cell:

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M = 484.22

SPSS: M = 484.22

However, when I try to get to the estimated marginal means, using the emmeans package:

eMeans <- emmeans(results$aov, ~ group | session | cue | flanker)

I run into descrepancies as compared to the Estimated Marginal Means table from the SPSS GLM output (for the same interactions), eg:

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M = 522.5643

SPSS: M = 484.22

It's been my understanding that the estimated marginal means should be the same as the descriptive means in this case, as I have not included any covariates. Am I mistaken in this? And if so, how come the two give different results?

Since the group sizes are unbalanced, I also redid the analyses above after making the groups of equal size. In that case the emmeans became:

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M =521.2954

SPSS: M = 482.426

So even with equal group sizes in both conditions, I end up with quite different means. Keep in mind that the rest of the statistics and the descriptive means áre equal between SPSS and R. What am I missing... ?



The plot thickens.. If I perform the ANOVA using the AFEX package:

results <- aov_ez(
    ,within=c("session", "cue", "flanker") 

and then take the emmeans again:

eMeans <- emmeans(results, ~ group | session | cue | flanker)

I suddenly get values much closer to that of SPSS (and the descriptive means)

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M = 484.08

SPSS: M = 484.22

So perhaps ezANOVA is doing something fishy somewhere?



answered 4 days ago rvl #1

I suggest you try this:

library(lme4)    ### I'm guessing you need to install this package first
mod <- lmer(rt ~ session + cue + flanker + (1|group),
            data = ant_rt_correct)

emm <- emmeans(mod, ~ session * cue * flanker)
pairs(emm, by = c("cue", "flanker")        # simple comparisons for session
pairs(emm, by = c("session", "flanker")    # simple comparisons for cue
pairs(emm, by = c("session", "cue")        # simple comparisons for flanker

This fits a mixed model with random intercepts for each group. It uses REML estimation, which is likely to be what SPSS uses.

In contrast, ezANOVA fits a fixed-effects model (no within factor at all), and aov_ez uses the aov function which produces an analysis that ignores the inter-block effects. Those make a difference especially with unbalanced data.

An alternative is to use afex::mixed, which in fact uses lme4::lmer to fit the model.

comments powered by Disqus