Music Sheets Interpreter

Offline sheet-music recognition for images and PDFs, with downloadable model weights, a Java decoder and a Python interface. Exports JSON, MIDI, MP3 and MusicXML.

Code and weights are Apache-2.0 licensed, including commercial and closed-source use. You do not need to publish your own code. Follow the license and notice requirements.

What it supports 🎹🎸🎻🎷

• Piano, Violin, Guitar 6- and 7 string tabs, ensemble pages, and independently barred staves.

• Standard notation: pitches, accidentals, chords, rests, ties, repeats and endings, tempo and key changes.

• Note-equals-number tempo marks, and note-equals-note metric modulations, including dotted pulses.

• Expressive previews: rit./rall. slowing, rite/ritenuto, fermata holds, breath pauses, sf/sfz/sfp attacks and paired tremolos. See the shared playback policy.

• Piano pedal brackets carry through to MIDI sustain, audio previews and MusicXML.

• Tuning headers, detached tab stems, partial beams, rests, dots, triplets, grace frets, tied continuations, hammer-on, pull-off, tapping, slide, bend, vibrato, and harmonic symbols. Missing tab rhythm is estimated. Labelled graphical bend arrows include quarter-tone amounts and return strokes; explicit whammy-bar arrows retain their pitch direction, and up/down strum marks order chord attacks. See gesture evidence and limits.

• JSON for integration, MIDI/MP3 for preview, and MusicXML for editing with retained clefs, slurs, articulations, ornaments and grace notes.

• Staff-owned arco, pizzicato and expression labels in MusicXML follow the first proved affected note's exported clock.

• Absolute dynamics at retained note columns use their proved written onsets in MusicXML, including staff ownership and edited timing.

• Explicit unpitched events and caller-selected previews: API examples.

• MusicXML reconstructs a concert-pitch score, not the original layout or tab placement. Guitar effects are text annotations in MusicXML.

Install and use

Requires Python 3.10 or 3.11 and JDK 17+. Build from source for the latest features; release downloads may be older.

git clone https://github.com/Luckolite/Music-Sheets-Interpreter.git
cd Music-Sheets-Interpreter
python -m venv .venv

Activate with .venv\Scripts\Activate.ps1 on Windows or source .venv/bin/activate on macOS/Linux, then run:

python scripts/build_java.py
python -m pip install ".[inference,pdf]"
sheet-interpreter score.pdf --output score.json --midi preview.mid --musicxml score.musicxml

Images work too. --midi and --musicxml are optional. Use --meter 3/4, --bpm 90, --key-fifths 2 or --pages 1,2,3 when needed. Default meter and tempo are 4/4 and 120 BPM.

Detected tempo changes report bpm in quarter notes per minute, including fractional values, and beatUnit as the printed pulse length in quarter notes. For example, eighth note = 163 reports bpm: 81.5, beatUnit: 0.5. MIDI and MusicXML use the quarter-note tempo directly. Decoded events include sourceNoteIndex, the zero-based head index within that page. Persistent instruction labels require matching source identities and consistent note clocks; older saved JSON without these identities keeps its prior omission of those labels. Everything runs locally after installation; no account or server is required.

MP3 audio previews

To export it, install FFmpeg on PATH or the optional bundled encoder:

python -m pip install ".[audio]"
sheet-interpreter score.pdf --output score.json --mp3 preview.mp3

You can export MIDI and MP3 together. --mp3-bitrate 192 sets the MP3 bitrate in kbps (default 192); --ffmpeg /path/to/ffmpeg selects an encoder explicitly. MP3 uses a basic built-in synthesized tone, not a realistic instrument soundfont. It shares MIDI's tempo changes, ties, tremolo, supported guitar effects and repeat navigation, including MIDI's current expressive limitations. Rendering is streamed in bounded blocks and limited to one hour. No soundfont, inference rerun or network is needed to export an already decoded document through the Python API. FFmpeg is a separate optional dependency with its own license; it is not committed or bundled in this repository's Apache-2.0 source/model artifacts.

Integration and model

Use Interpreter and write_musicxml from Python, or SheetInterpreter.analyze() from Java. See API examples and output format. An optional native Java decoding service supports bounded, source-matched geometry and analysis requests from local workers. The Python reader runs bundled-model OCR automatically on images and scanned PDFs. The separate shared Java OCR pipeline and optional ONNX binding remain available for cross-platform evaluation. The optional desktop ONNX segmentation adapter uses the pinned v3 conversion without changing the default Python model.

The bundled v4 model comes from our own synthetic training lineage, without pretrained weights or commercial score scans. See the model card, evaluation, training guide and source origins.

Development

python scripts/build_java.py
python scripts/test_java.py
python -m unittest discover -s tests
python scripts/smoke_test.py

Use python scripts/test_java.py --jobs 4 in place of the serial Java test command to run every class once across four isolated JVMs. Each parallel JVM defaults to a 512 MiB heap; per-batch logs and results are saved under build/test-parallel. A failed batch makes the command fail.

Bug reports and contributions are welcome. Use shareable examples and keep private logs and commercial scores out of public posts. See CONTRIBUTING.md.

Pickup caller integration regression

After building the Java core, run the original numeric/content cases against the actual standalone pickup call:

python scripts/check_pickup_ingress.py --public . --classpath src/sheet_interpreter/interpreter.jar --out build/pickup-ingress

Use a new output directory on each run. Maintainers can add --app <app-worktree> and use its matching core classpath to check the exact Android and native caller expressions too. This compiles source slices with current helper sources; it does not run Android, image recognition or a packaged worker. The fixtures contain no score images or private data.