PsyR: An R package for better inference in multivariate statistical analysis
Kelly Garner (Mattermost: kels)
Christopher Nolan (Mattermost: cnolan)
Brainhack Aus
We consistently use massive data sets across neuroscience and psychology. The routine gathering of big data requires that we are well equipped with tools that allow us to conduct appropriate multivariate statistics.
Multivariate statistical analysis typically follows a two stage procedure, an omnibus test of the global null hypothesis followed by post-hoc tests of specific effects. It is not well known that under certain circumstances this leads to a drastically inflated rate of type 1 error. It is even less well known that this procedure can also lead to an even lessor known type IV error (incorrect interpretation of a correctly rejected hypothesis)!
It is possible to avoid these dragons by using an alternative procedure where all inferences are derived from simultaneous confidence intervals (SCIs) on contrasts of interest. This approach provides interval inferences on effect sizes and it also ensures that the familywise type 1 error rate associated with directional inferences (the inferences usually derived from tests of null hypotheses) is controlled at alpha. One piece of software (PSY) can produce SCIs appropriate for planned analyses (where contrasts are defined independently of the data) and for more flexible analyses where contrasts are defined on a post hoc basis. However, this software is only available for use on windows and cannot be scripted into reproducible workflows.
We have been building an R package that implements the functions of PSY, and to make this method of statistical inference available to the masses! We are ready for you to play with some of the functions to analyse your own data, and create Quarto documents that document your workflow, to help future users. If you are feeling hacky enough, we could also use help developing further the R package, which is around 80% finished right now.
https://github.com/kel-github/PSY2R - this has all the code we used to get started. I’ll share the package code at the event https://doi.org/10.1177/0013164402062002001 - this is the paper that defines the functionality of Psy. https://r-pkgs.org/ - Hadley Wickham’s bible for building R packages
Bring your own data and use the functions we have already written. This will help us learn how user friendly they are and what can be done to improve interaction with the function. Document own data analysis in quarto documents to help future users Contribute to the R package, particularly with higher level functions that handle data wrangling and standardisation Build a GitHub website for the package
issue one: read [this paper](https://doi.org/10.1177/0013164402062002001 to get an overview of what Psy does)
issue two: find a dataset that requires an ANOVA analysis, share it with the team and we’ll talk you through how to use Psy to analyse it
issue three: If not already, get comfortable with quarto documents - https://quarto.org/
issue four: Go through the exercises that show you how to create an R package in chapter 1 of this book .
https://mattermost.brainhack.org/brainhack/channels/psy2r
No response
Participants will learn about multivariate data analysis:
No response
4
Project contributors will be listed in the project ReadMe, and as authors on any help documents
Leave this text if you don’t have an image yet.
coding_methods, documentation, pipeline_development
1_basic structure
statistical_modelling
other
R
behavioral, eye_tracking
1_commit_push, 2_branches_PRs
No response
Hi @brainhackorg/project-monitors my project is ready!