lytk

lytk reads and writes symbolic music, and turns it into training data. Scores in LilyPond, MusicXML, MIDI, ABC and Humdrum **kern go through one internal representation, so any format converts to any other. The same scores come out as NumPy note arrays, piano rolls and event sequences, ready for PyTorch or TensorFlow. The core is written in Rust; you use it from Python or from the command line.

Preview release (0.x). Conversion, transforms and the ML pipeline are tested and ready to use. The API can still change before 1.0: notation marks (articulations, ornaments, dynamics) will move from strings to enums.

Why lytk

• LilyPond is a first-class format. lytk reads real LilyPond files, not a subset: variables, \relative, piano scores with several voices per staff, repeats and voltas, cadenzas, lyrics, chord names and figured bass. It writes them back out too, in any of LilyPond's 12 note-name languages.
• Conversions are measured, not assumed. Every round trip (LilyPond, MusicXML, ABC, **kern, MIDI) is checked in CI against a test corpus for notes (onsets, durations, sounding pitches), lyrics, chord symbols, dynamics and clefs at their places; stems and beams against the engravers that wrote the files; LilyPond output against LilyPond itself. The results can only improve from one release to the next (the numbers).
• Built for machine learning. Encoders for note arrays, piano rolls and Performance-RNN event sequences, objective metrics from the muspy family, datasets over folders or JSON Lines records with deterministic (or the records' own) splits and caching, and padded data loaders for PyTorch and TensorFlow.
• Fast. A Rust core with prebuilt wheels, and a batch converter that uses every core. It transposes LilyPond about 50× faster than python-ly.

Install

pip install lytk                  # every format, transform and representation
pip install "lytk[torch]"         # + PyTorch datasets and data loaders
pip install "lytk[tensorflow]"    # + tf.data datasets and data loaders
pip install "lytk[eval]"          # + generation-evaluation metrics (scipy, FMD)
pip install "lytk[all]"           # everything above

Wheels are prebuilt for Linux, macOS and Windows and cover CPython 3.10 and newer, so no Rust toolchain is needed. PyTorch and TensorFlow are optional and only imported when you ask for a loader.

Quick start

import lytk

score = lytk.from_musicxml("input.xml")      # or from_lilypond, from_midi, from_abc, from_humdrum
score = lytk.transpose(score, semitones=3)   # also invert, retrograde, change_language
lytk.to_lilypond(score, "output.ly")         # or to_musicxml, to_midi, to_abc, to_humdrum
abc = lytk.to_abc(score)                     # without a path, writers return the text

Check LilyPond before you rely on it:

for d in lytk.check_lilypond(text, semantic=True):   # errors and warnings, in source order
    print(d)                                         # 3:12: error: missing `}` [missing-token]
score = lytk.from_lilypond("score.ly", strict=True)  # raises lytk.LilyPondSyntaxError on an error
score.diagnostics                                    # what the reading reported, either way
score.header                                         # every \header field, as a dict
fields = lytk.header_fields(text)                    # …or read from the text: key, value, span

A reader raises lytk.ParseError (a ValueError) when its input cannot be read, and OSError when the file cannot be opened.

Turn a score into arrays:

doc = score.to_music_document()
notes = lytk.to_note_array(doc)        # (N, 4): onset, duration, pitch, velocity
roll = lytk.to_piano_roll(doc)         # (T, 128)
events = lytk.to_event_sequence(doc)   # Performance-RNN event codes
stats = lytk.compute_metrics(doc)      # pitch-class entropy, polyphony, scale consistency, …

Each encoder has an inverse (from_note_array, …). The arrays are ordinary NumPy arrays, and NumPy supports DLPack, so PyTorch, JAX and CuPy can use them without copying:

import torch
tensor = torch.from_dlpack(lytk.to_piano_roll(doc))   # shares the buffer

Build a dataset from a folder of scores in any mix of formats:

from lytk.datasets import FolderDataset

data = FolderDataset("corpus/", cache_dir=".cache")
train, val, test = data.split((0.8, 0.1, 0.1), seed=0)

loader = train.to_pytorch_dataloader("event_sequence", batch_size=32, shuffle=True)
for events, lengths in loader:         # padded batch plus each item's true length
    ...

Scores differ in length, so each batch is padded. The true lengths come back alongside it because 0 is a valid event, pitch and velocity, so padding alone can't tell you where a score ends. to_tensorflow_dataloader works the same way.

A corpus kept as JSON Lines records (id, text, metadata) keeps its ids and its own splits:

from lytk.datasets import RecordsDataset

data = RecordsDataset.from_jsonl("scores.jsonl", split_field="split", on_error="skip")
splits = data.split()                  # {"train": …, "valid": …, "test": …}, as recorded
loader = splits["train"].to_pytorch_dataloader("note_array", batch_size=32, return_ids=True)
for arrays, lengths, ids in loader:
    ...

The package also installs a lytk command:

lytk convert input.xml -o output.ly           # formats are taken from the extensions
lytk convert corpus/ -o out/ -f xml -j 8      # a whole folder, in parallel
lytk transpose input.ly -s 3 -o up.ly
lytk flatten score.ly -o flat.ly              # inline every \include
lytk check score.ly --semantic                # report the errors of LilyPond files
lytk info input.mxl

It also inverts, reverses, changes note-name languages, compares scores (lytk diff) and runs JSON batch jobs; lytk --help lists every command and docs/cli.md describes them.

Formats

docs/import-export.md lists what each reader and writer keeps. MusicXML is the most complete. MIDI export plays a score the way LilyPond's own MIDI does (dynamics, articulations, grace notes, repeats, pedal, lyrics); slurs and ornaments are not performed.

Not there yet

• Tablature, unpitched notes and fretboard diagrams (LilyPond drums are read as their General MIDI keys on a percussion staff).
• Performed MIDI is quantized less reliably when its tempo is far from the file's own, when chords are released unevenly, or at a low resolution.
• Non-traditional key signatures, cross-staff notes (\change Staff), and Humdrum spine splits (*^, *v). Files that use spine splits are rejected with an error.
• The ABC and **kern writers produce one stream per staff, so two voices sharing a staff are not kept separate.
• MEI is planned after 1.0.

Documentation

• Python API reference
• Command line
• What each format reads and writes
• Design and internals
• Building from source and running the tests
• Changelog and roadmap

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md, and SECURITY.md for reporting vulnerabilities.

Acknowledgements

lytk builds on a lot of prior work in music notation software. Its LilyPond parser uses the tree-sitter-lilypond grammar by Nathan Whetsell (MIT, notice in src/tree-sitter/LICENSE). Its design also owes a lot to LilyPond, python-ly, music21, muspy, abjad, symusic and MuseScore.

License

MIT, see LICENSE. The test fixtures are third-party scores under their own terms (tests/fixtures/README.md).