Site icon R-bloggers

The end of errors in ANOVA reporting

Dominique Makowski

[This article was first published on Dominique Makowski, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Psychology is still (unfortunately) massively using analysis of variance (ANOVA). Despite its relative simplicity, I am very often confronted to errors in its reporting, for instance in student’s theses or manuscripts. Beyond the incomplete, uncomprehensible or just wrong reporting, one can find a tremendous amount of genuine errors (that could influence the results and their intepretation), even in published papers! (See the excellent statcheck to quickly check the stats of a paper). This error proneness can be at least partially explained by the fact that copy/pasting the (appropriate) values of any statistical software and formatting them textually is a very annoying process.

How to end it?

We believe that this could be solved (at least, partially) by the default implementation of current best practices of statistical reporting. A tool that automatically transforms a statistical result into a copy/pastable text. Of course, this automation cannot be suitable for each and every advanced usage, but would probably be satisfying for a substantial proportion of use cases. Implementing this unified, end-user oriented pipeline is the goal of the psycho package.

Fit an anova

Let’s start by doing a traditional ANOVA with adjusting (the ability to flexibly regulate one’s emotions) as dependent variable, and sex and salary as categorical predictors.

# devtools::install_github("neuropsychology/psycho.R")  # Install the latest psycho version
library(psycho)

df <- psycho::affective  # load a dataset available in the psycho package

aov_results <- aov(Adjusting ~ Sex * Salary, data=df)  # Fit the ANOVA
summary(aov_results)  # Inspect the results

             Df Sum Sq Mean Sq F value   Pr(>F)    
Sex           1   35.9   35.94  18.162 2.25e-05 ***
Salary        2    9.4    4.70   2.376   0.0936 .  
Sex:Salary    2    3.0    1.51   0.761   0.4674    
Residuals   859 1699.9    1.98                     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
386 observations deleted due to missingness

APA formatted output

The psycho package include a simple function, analyze() that can be applied to an ANOVA object to format its content.

analyze(aov_results)

   - The effect of Sex is significant (F(1, 859) = 18.16, p < .001) and can be considered as small (Partial Omega-squared = 0.019).
   - The effect of Salary is not significant (F(2, 859) = 2.38, p = 0.09°) and can be considered as very small (Partial Omega-squared = 0.0032).
   - The interaction between Sex and Salary is not significant (F(2, 859) = 0.76, p > .1) and can be considered as very small (Partial Omega-squared = 0).

It formats the results, computes the partial omega-squared as an index of effect size (better than the eta2, see Levine et al. 2002, Pierce et al. 2004) as well as its interpretation and presents the results in a APA-compatible way.

Correlations, t-tests, regressions…

Note that the analyze() method also exists for other statistical procudures, such as correlations, t-tests and regressions.

Evolution

Of course, these reporting standards should change, depending on new expert recommandations or official guidelines. The goal of this package is to flexibly adaptive to new changes and good practices evolution. Therefore, if you have any advices, opinions or such, we encourage you to either let us know by opening an issue, or even better, try to implement them yourself by contributing to the code.

Credits

This package helped you? Don’t forget to cite the various packages you used 🙂

You can cite psycho as follows:

On similar topics

To leave a comment for the author, please follow the link and comment on their blog: Dominique Makowski.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.