notochord 0.8.0

Notochord is a real-time neural network model for MIDI performances.

Notochord (Documentation | Paper | Video)

Notochord is a neural network model for MIDI performances. This package contains the training and inference model implemented in pytorch, as well as interactive MIDI processing apps using iipyper.

API Reference

Getting Started

We recommend using uv to run notochord, install it as a tool, or create a project incorporating it.

Install fluidsynth (optional)

fluidsynth is a General MIDI synthesizer which you can install from the package manager. On macOS:

brew install fluidsynth

fluidsynth needs a soundfont to run, like this one: https://drive.google.com/file/d/1-cwBWZIYYTxFwzcWFaoGA7Kjx5SEjVAa/view

You can run fluidsynth in a terminal. For example, fluidsynth -v -o midi.portname="fluidsynth" -o synth.midi-bank-select=mma ~/'Downloads/soundfonts/Timbres of Heaven (XGM) 4.00(G).sf2'

try out the homunculus MIDI app right now

uvx notochord homunculus

persistently install notochord

uv tool install notochord
notochord --help

create a Python project using notochord

uv init my-project
cd my-project
uv add notochord
.venv/bin/python -m notochord --help

without uv

Using your python environment manager of choice (e.g. virtualenv, conda), make a new environment with a Python version between 3.10-3.13. Then pip install notochord.

developing

For developing notochord, see our dev repo

Notochord Homunculus

Notochord includes several iipyper apps which can be run in a terminal. They have a clickable text-mode user interface and connect directly to MIDI ports, so you can wire them up to your controllers, DAW, etc.

The homunculus provides a text-based graphical interface to manage multiple input, harmonizing or autonomous notochord voices:

notochord homunculus

You can set the MIDI in and out ports with --midi-in and --midi-out. If you use a General MIDI synthesizer like fluidsynth, you can add --send-pc to also send program change messages. More information in the Homunculus docs, or run notochord homunculus --help

If you are using fluidsynth as above, try:

notochord homunculus --send-pc --midi-out fluidsynth --thru

Note: on windows, there are no virtual MIDI ports and no system MIDI loopback, so you may need to attach some MIDI devices or run a loopback driver like loopMIDI before starting the app.

If you pass homunculus a MIDI file using the --midi-prompt flag, it will play as if continuing after the end of that file.

Adding the --punch-in flag will automatically switch voices to input mode when MIDI is received and back to auto after some time passes.

Python API

See the docs for Notochord.reset, Notochord.feed and Notochord.sample (or older Notochord.query) for the low-level Notochord inference API which can be used from Python code. notochord/app/simple_harmonizer.py provides a minimal example of how to build an interactive app.

OSC server

You can also expose the inference API over Open Sound Control:

notochord server

this will run notochord and listen continously for OSC messages.

Tidal interface

see notochord/tidalcycles in iil-examples repo (updated examples coming soon):

add Notochord.hs to your tidal boot file. Probably replace the tidal <- startTidal line with something like:

:script ~/iil-examples/notochord/tidalcycles/Notochord.hs

let sdOscMap = (superdirtTarget, [superdirtShape])
let oscMap = [sdOscMap,ncOscMap]

tidal <- startStream defaultConfig {cFrameTimespan = 1/240} oscMap

In a terminal, start the python server as described above.

In Supercollider, step through examples/notochord/tidalcycles/tidal-notochord-demo.scd which will receive from Tidal, talk to the python server, and send MIDI on to a synthesizer. There are two options, either send to fluidsynth to synthesize General MIDI, or specify your own mapping of instruments to channels and send on to your own DAW or synth.

Train your own Notochord model (GPU recommended)

Select the train option when installing, e.g. uv tool install notochord[train].

preprocess data

notochord prep --data_path /path/to/midi/files --dest_path /path/to/data/storage

This will process all MIDI files under a directory, converting them to torch tensors which notochord can train on. The built-in preprocessing assumes a dataset of General MIDI files, like the Lakh MIDI dataset. If you use your own MIDI files, you likely want to label parts using MIDI program change messages (different instruments for each channel). Only ProgramChange, NoteOn and NoteOff events are processed by notochord.

launch a training job

notochord train my-model --data_dir /path/to/data/storage --log_dir /path/for/tensorboard/logs --model_dir /path/for/checkpoints --results_dir /path/for/other/logs

The above will train a new notochord model from scratch. By adding the --model argument, you can set model hyperparameters (as documented in Notochord.__init__). For example, --model '{rnn_hidden:512, rnn_layers:1, mlp_layers:1}' would train a smaller model than the default.

If using your own dataset, you may want to turn off data augmentation options with --aug-speed 0, --aug-transpose 0 or --aug-remap False.

Training progress can be monitored via tensorboard:

tensorboard --logdir /path/for/tensorboard/logs

The most important value is valid/loss. As long as it decreases, the model should continue to improve. Training will continue until the job is killed with ctrl+C.

resuming and fine-tuning

You can resume training from an existing checkpoint by adding --resume --checkpoint /path/to/model/checkpoint. It is also possible to initialize training from a checkpoint without carrying over optimizer states etc from the original training, by leaving out the --resume flag. For fine-tuning on small datasets, consider adding --freeze-rnn. Training options are documented under Trainer.__init__

To fine the latest base notochord model, you can run notochord homunculus at least once to download it, then use notochord files to find the downloaded model.

use your custom model

You can load a custom model in the Python API using Notchord.from_checkpoint or use it with homunculus using e.g. notochord homunculus --checkpoint /path/for/checkpoints/my-model/XXX.ckpt