Psy2R: Developing an R package for better inference in multivariate statistical analysis
Title
Psy2R: Developing an R package for better inference in multivariate statistical analysis
Leaders
Kelly Garner Github: @kel-github Mattermost: @kels
Collaborators
Kevin Bird,
Melanie Gleitzman,
Sonny Li,
Christopher Nolan Github: @crnolan Mattermost: @cnolan
Brainhack Global 2023 Event
Brainhack Australasia
Project Description
We consistently use massive data sets across neuroscience and psychology. The routine gathering of big data requires that we are well equipped with tools that allow us to conduct appropriate multivariate statistics.
Multivariate statistical analysis typically follows a two stage procedure, an omnibus test of the global null hypothesis followed by post-hoc tests of specific effects. It is not well known that under certain circumstances this leads to a drastically inflated rate of type 1 error. It is even less well known that this procedure can also lead to an even lessor known type IV error (incorrect interpretation of a correctly rejected hypothesis)!
It is possible to avoid these dragons by using an alternative procedure where all inferences are derived from simultaneous confidence intervals (SCIs) on contrasts of interest. This approach provides interval inferences on effect sizes and it also ensures that the familywise type 1 error rate associated with directional inferences (the inferences usually derived from tests of null hypotheses) is controlled at alpha. One piece of software (PSY) can produce SCIs appropriate for planned analyses (where contrasts are defined independently of the data) and for more flexible analyses where contrasts are defined on a post hoc basis. However, this software is only available for use on windows and cannot be scripted into reproducible workflows.
Our goal is to build an R package that implements the functions of PSY, and to make this method of statistical inference available to the masses!
Link to project repository/sources
TBC
Goals for Brainhack Global
Milestones
- Implement analyses that can be done in PSY in R [intermediate]
- Convert R implementation into generalisable functions [intermediate/advanced]
- Outline the math involved in computing critical values for STP tests under the null hypothesis [advanced]
- Interpret the PASCAL code that underpins the computation of critical values in PSY [advanced]
- Investigate the R world for a comprehensive list of packages that share some functionality with PSY [beginner]
- Test R functions and implement own analysis [beginner]
- Interpret what you think our R code does, to help us write more readable code [beginner]
- Document the PSY software functions and write documentation for the new R functions [beginner]
- Create project management board (e.g. trello) to keep us on track! [beginner]
Good first issues
Implement analyses that can be done in PSY in R
Document existing PSY software functions
Investigate the R world for a comprehensive list of packages that share some functionality with PSY
Create project management board (e.g. trello) to keep us on track!
Create github repo for the project
Communication channels
https://mattermost.brainhack.org/brainhack/channels/psy2r
Skills
- R: all levels
- stats/math: all levels
- github: all levels
- comp sci: intermediate/advanced
Onboarding documentation
No response
What will participants learn?
You’ll learn more about statistical analysis of big datasets, collaborative coding using R and github, and hopefully a bit about package development and coding for other users.
Data to use
No response
Number of collaborators
more
Credit to collaborators
Project contributors will be listed on the github repository’s README.md
Image
Type
coding_methods, data_management, documentation, method_development
Development status
0_concept_no_content
Topic
reproducible_scientific_methods, statistical_modelling
Tools
other
Programming language
C++, documentation, R, other
Modalities
behavioral, DWI, ECOG, EEG, eye_tracking, fMRI, MEG, MRI, PET
Git skills
1_commit_push
Anything else?
No response
Things to do after the project is submitted and ready to review.
- Add a comment below the main post of your issue saying:
Hi @brainhackorg/project-monitors my project is ready! - Twitter-sized summary of your project pitch.