Brainharmonic - Generating music from brain signal data using deep learning

Title

Brainharmonic - Generating music from brain signal data using deep learning

Leaders

• Mariano Cabezas
• Aria Nguyen

Collaborators

No response

Brainhack Global 2021 Event

Brainhack Australasia

Project Description

This project aims to develop a tool to generate music from brain signals using deep learning models. There has been a lot of work on generating new music from a large collection of music using deep AI models, as well as some work on generating music from brain EEG/fMRI signal through algorithmic or rule based approaches. However there has not been work done on using deep learning models to generate music from EEG/fMRI signal.

In this project, we will use deep generative models to allow EEG/fMRI data to be used as an input to generate music. We will also look into linking brain signal from different regions of the brain to different instruments to create a symphony. More details of the design can be found on our Github page.

Link to project repository/sources

https://github.com/marianocabezas/brainharmonic

Goals for Brainhack Global

Develop a tool to generate songs from EGG/fMRI brain signals using deep learning

Good first issues

1. Issue one: Develop a generative deep learning model to generate songs from a latent space
2. Issue two: Develop a deep learning model to translate EEG/fMRI sequences from similar subjects into a latent space

Communication channels

https://mattermost.brainhack.org/brainhack/channels/brainharmonic

Skills

• Python: intermediate
• Pytorch: beginner
• EEG/fMRI data processing: beginner

Onboarding documentation

No response

What will participants learn?

• EEG/fRMI data preprocessing
• Deep learning generative models
• Clustering/classification modelling techniques
• Existing music-to-music deep AI frameworks

Data to use

No response

Number of collaborators

2

Credit to collaborators

Project contributors are credited on Readme file on project Github page

Image

Type

coding_methods, data_management, method_development, pipeline_development

Development status

0_concept_no_content

Topic

deep_learning, information_theory, machine_learning

Tools

fMRIPrep, Freesurfer

Programming language

Python

Modalities

EEG, fMRI

Git skills

1_commit_push, 2_branches_PRs

Anything else?

No response

Things to do after the project is submitted.

• Twitter-sized summary of your project pitch.