# Discrepancy emmeans in R (using ezAnova) vs estimated marginal means in SPSS

Kerwin Olfers Source

So this is a bit of a hail mary, but I'm hoping someone here has encountered this before. I recently switched from SPSS to R, and I'm now trying to do a mixed-model ANOVA. Since I'm not confident in my R skills yet, I use the exact same dataset in SPSS to compare my results.

I have a dataset with

dv = RT

within = Session (2 levels), Cue (3 levels), Flanker (2 levels)

between = Group(3 levels).

no covariates.

unequal number of participants per group level (25,25,23)

In R I'm using the ezAnova package to do the mixed-model anova:

``````results <- ezANOVA(
data = ant_rt_correct
, wid = subject
, dv = rt
, between = group
, within = .(session, cue, flanker)
, detailed = T
, type = 3
, return_aov = T
)
``````

In SPSS I use the following GLM:

``````GLM rt.1.center.congruent rt.1.center.incongruent rt.1.no.congruent rt.1.no.incongruent
rt.1.spatial.congruent rt.1.spatial.incongruent rt.2.center.congruent rt.2.center.incongruent
rt.2.no.congruent rt.2.no.incongruent rt.2.spatial.congruent rt.2.spatial.incongruent BY group
/WSFACTOR=session 2 Polynomial cue 3 Polynomial flanker 2 Polynomial
/METHOD=SSTYPE(3)
/EMMEANS=TABLES(group*session*cue*flanker)
/PRINT=DESCRIPTIVE
/CRITERIA=ALPHA(.05)
/WSDESIGN=session cue flanker session*cue session*flanker cue*flanker session*cue*flanker
/DESIGN=group.
``````

The results of which line up great, ie:

R: Session F(1,70) = 46.123 p = .000

SPSS: Session F(1,70) = 46.123 p = .000

I also ask for the means per cell using:

``````descMeans <- ezStats(
data = ant_rt_correct
, wid = subject
, dv = rt
, between = group
, within = .(session, cue, flanker) #,cue,flanker)
, within_full = .(location,direction)
, type = 3
)
``````

Which again line up perfectly with the descriptives from SPSS, e.g. for the cell:

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M = 484.22

SPSS: M = 484.22

However, when I try to get to the estimated marginal means, using the emmeans package:

``````eMeans <- emmeans(results\$aov, ~ group | session | cue | flanker)
``````

I run into descrepancies as compared to the Estimated Marginal Means table from the SPSS GLM output (for the same interactions), eg:

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M = 522.5643

SPSS: M = 484.22

It's been my understanding that the estimated marginal means should be the same as the descriptive means in this case, as I have not included any covariates. Am I mistaken in this? And if so, how come the two give different results?

Since the group sizes are unbalanced, I also redid the analyses above after making the groups of equal size. In that case the emmeans became:

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M =521.2954

SPSS: M = 482.426

So even with equal group sizes in both conditions, I end up with quite different means. Keep in mind that the rest of the statistics and the descriptive means áre equal between SPSS and R. What am I missing... ?

Thanks!

EDIT:

The plot thickens.. If I perform the ANOVA using the AFEX package:

``````results <- aov_ez(
"subject"
,"rt"
,ant_rt_correct
,between=c("group")
,within=c("session", "cue", "flanker")
)
)
``````

and then take the emmeans again:

``````eMeans <- emmeans(results, ~ group | session | cue | flanker)
``````

I suddenly get values much closer to that of SPSS (and the descriptive means)

Group(1) - Session(1) - Cue(center) - Flanker(1)

R: M = 484.08

SPSS: M = 484.22

So perhaps ezANOVA is doing something fishy somewhere?

rstatisticsspss

answered 6 months ago rvl #1

I suggest you try this:

``````library(lme4)    ### I'm guessing you need to install this package first
mod <- lmer(rt ~ session + cue + flanker + (1|group),
data = ant_rt_correct)

library(emmeans)
emm <- emmeans(mod, ~ session * cue * flanker)
pairs(emm, by = c("cue", "flanker")        # simple comparisons for session
pairs(emm, by = c("session", "flanker")    # simple comparisons for cue
pairs(emm, by = c("session", "cue")        # simple comparisons for flanker
``````

This fits a mixed model with random intercepts for each group. It uses REML estimation, which is likely to be what SPSS uses.

In contrast, `ezANOVA` fits a fixed-effects model (no within factor at all), and `aov_ez` uses the `aov` function which produces an analysis that ignores the inter-block effects. Those make a difference especially with unbalanced data.

An alternative is to use `afex::mixed`, which in fact uses `lme4::lmer` to fit the model.