libretta 0.1.1

Audio analysis pipeline: stem separation, MIDI transcription, spectral analysis, chord detection, ghost frequency diagnostics, and spectrogram visualization.

libretta

Audio analysis pipeline that makes AI understand music. Stem separation, MIDI transcription, spectral analysis, chord detection, ghost frequency diagnostics, and spectrogram visualization, all from one command.

libretta takes an audio file, tears it apart, measures everything, and gives you structured data that a language model can reason over. Think of it as librosa's successor that actually tells you what's happening in a song instead of just handing you raw numbers.

What it does

1. Stem separation (Demucs) splits a mix into vocals, drums, bass, and other
2. MIDI transcription (Basic Pitch) detects notes and exports MIDI files with pitch bends
3. Feature extraction (librosa) pulls tempo, key, chroma, spectral features, dynamics, MFCCs, beat timing, harmonic/percussive ratio
4. Chord detection estimates chords at each beat using chroma template matching
5. Ghost frequency detection finds inaudible subsonic/ultrasonic energy eating your headroom
6. Spectrogram rendering generates mel spectrograms, chromagrams, piano rolls, waveforms, and frequency spectrum plots

Each step runs on both the full mix and each individual stem for maximum accuracy.

Install

pip install libretta

For full features (stem separation + better MIDI transcription):

pip install libretta[all]

On Windows, if basic-pitch gives you trouble:

pip install onnxruntime
pip install "basic-pitch==0.3.3" --no-build-isolation

For GPU acceleration (recommended), install PyTorch with CUDA:

pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu121

Usage

Command line

# full analysis
libretta song.mp3

# skip stem separation (faster, just analyze the mix)
libretta song.wav --no-stems

# skip spectrogram images
libretta song.flac --no-images

# use the fine-tuned demucs model (better quality, slower)
libretta song.mp3 --model htdemucs_ft

# force CPU
libretta song.mp3 --cpu

# custom output directory
libretta song.mp3 -o ./my_analysis

Python API

from libretta import analyze, separate_stems, render_spectrograms

# analyze a single file
result = analyze("song.mp3", tag="mix")
print(result["key"])       # "F major"
print(result["bpm"])       # 120.5
print(result["chords"])    # [{"time": 0.0, "chord": "Fmaj", "conf": 0.89}, ...]
print(result["notes"])     # [{"start": 0.5, "end": 1.2, "pitch": 69, "name": "A4", "vel": 0.7}, ...]

# separate stems
stems = separate_stems("song.mp3", "./stems")
# returns {"drums": "./stems/drums.wav", "bass": "./stems/bass.wav", ...}

# render spectrograms
render_spectrograms("song.mp3", "mix", "./images", notes=result["notes"])

Output

Running libretta song.mp3 creates:

analysis_song/
  analysis.json          # all data, structured, feedable to an LLM
  report.txt             # human readable summary
  stems/
    drums.wav
    bass.wav
    other.wav
    vocals.wav
    other.mid            # MIDI transcription per stem
    bass.mid
    vocals.mid
  images/
    mix_overview.png     # mel spectrogram + chromagram + piano roll + waveform
    mix_spectrum.png     # frequency spectrum with ghost frequency zones
    drums_overview.png
    drums_spectrum.png
    ...

Ghost frequencies

libretta detects inaudible subsonic (below 20Hz) and ultrasonic (above 18kHz) energy that eats headroom and confuses limiters. It filters these out before analysis so the readings are accurate, but flags them in the report so you know your mix has a problem to fix.

Silent stems (like when Demucs splits a solo piano into "drums" and gets basically nothing) are automatically detected via RMS threshold and marked as unreliable.

Requirements

• Python 3.10+
• librosa, numpy, scipy, soundfile, matplotlib, torch, torchaudio (core)
• demucs (optional, for stem separation)
• basic-pitch + onnxruntime (optional, for MIDI transcription, falls back to pyin)
• CUDA GPU recommended but not required

License

MIT