Bayes and Bach: An RNN-powered affair
Title
Bayes and Bach: An RNN-powered affair
Leaders
Alejandro Tabas (mattermost: @atabas)
Collaborators
No response
Brainhack Global 2024 Event
Brainhack Donostia
Project Description
We want to do a proof-of-concept: test whether artificial recurrent neural networks (RNN) learn to predict the next tone when trying to perceive music in a noisy environment.
This is an accessible project that does not require an extensive coding expertise. Participants that want to contribute with coding will be recommended to do this in python, although data retrieval can be done in any other language. Participants who do not wish to contribute with code can contribute either with a) their musical expertise in Data Preprocessing and during the interpretation of the Testing, or b) their patience and web-browsing skills to manual data retrieval.
We will start by collecting a lot of Bach’s compositions. We will transform each composition into a sequence of tokens: each token will be one note or several notes (i.e., a chord) that were meant to be played together in the original composition. For this proof-of-concept we will ignore information about the duration or loudness of each note/chord. For each sequence, we will generate two items: a ground-truth (the sequence itself), and a set of observations (the sequence plus some noise that will corrupt part of the information).
We will then present the observations to the RNN and train its weights so that, for each of the observation sequences, the RNN is able to extract the ground-truth sequence. In other words: we will train the RNNs to perceive the original musical sequence usign only the noisy observations.
After the training we will test whether the RNN is encoding, in its internal states, a set of predictions on the next token. If that is the case, we will conclude that the RNN is actively attempting to predict the next item to implement perception.
The results will help us generating hypotheses on how the brain decodes music. The neuroscientific community will be able to use these hypotheses to inform the design of experiments IRL brains.
More details and resources in the github repository
Link to project repository/sources
https://github.com/qtabs/BayesPlusBach/ https://mattermost.brainhack.org/brainhack/channels/bh-donostia-2024--bayes-3-bach
Goals for Brainhack Global
Milestones:
- A web scrapper that collects MIDI files of Bach’s compositions
- A collection of MIDI files of Bach’s compositions
- A MIDI-to-data pipeline that transforms MIDI files in pairs of (ground-truth, observation) that can be used for training
- Several RNN architectures designed to retrieve the ground truth from noisy observations of sequences of tones/chords
- The trained RNN(s)
- Validation/refutation of the starting hypothesis
Good first issues
- Automatise the retrieval of Bach’s MIDIs
- Automatise the encoding of a MIDI into a training sample for the RNN
- Build an RNN architecture to predict the training samples.
Communication channels
https://mattermost.brainhack.org/brainhack/channels/bh-donostia-2024--bayes-3-bach
Skills
Python: all levels
Onboarding documentation
https://github.com/qtabs/BayesPlusBach/blob/main/CONTRIBUTING.md
What will participants learn?
Web scrapping with python Training RNNs with pytorch
Data to use
Resources on MIDI files: This website could be a good place to start. This other website has a lot of links with potentially better quality MIDI files. The better the quality of the MIDI, the easier the work of the preprocesser(s)!
Resources on data scrappers: I used this tutorial when I needed to implemet a scraper in the past. This other tutorial also looks informative and easy to follow.
Number of collaborators
3
Credit to collaborators
Contributors will be listed in the contributors textfile.
Image
Type
method_development
Development status
0_concept_no_content
Topic
machine_learning
Tools
other
Programming language
Python
Modalities
behavioral
Git skills
0_no_git_skills
Anything else?
No response
Things to do after the project is submitted and ready to review.
- Add a comment below the main post of your issue saying:
Hi @brainhackorg/project-monitors my project is ready! - Twitter-sized summary of your project pitch.