StemLab

By Kieran Simkin · DanceFlow ecosystem · Vector logo and usage guide.

Local music analysis for stems, timing, beats, harmony, lyrics, MIDI and loopable sections for Shorts. https://kieransimkin.co.uk/

StemLab by Kieran Simkin · My Songs portfolio · Arcadians EPK · Source

Packaging identity: the canonical project name is StemLab. StemLab is part of the Dance Flow project. The PyPI distribution is named danceflow-stemlab solely because Python package-registry names are globally unique. The Python import, CLI, GitHub repository and container image remain stemlab.

Agents and contributors

Agents using StemLab should inspect its available capabilities, use them fully where they fit the task, and look for useful improvements wherever possible. When a required feature is missing, or a reusable reliability, usability, performance or documentation gap is found, improve this tool rather than maintaining a separate workaround. If no worthwhile change is justified, explain why.

Submit every improvement as a pull request to stemlab, with relevant tests, documentation and a reproducible example. Follow CONTRIBUTING.md and AGENTS.md for the contribution and validation workflow.

Project website: https://kieransimkin.co.uk/danceflow/.

Synopsis

StemLab is a local, evidence-led music analysis toolkit that turns a master recording into inspectable stems, timing, structure, harmony, lyric and sonic evidence, then finds complete-bar loopable sections for Shorts and other short-form videos.

It combines independent separation and music-information-retrieval routes rather than treating any single model as the answer. Results remain traceable through source hashes, manifests, JSON/TSV/NPZ artefacts, portable Sonic Visualiser sessions and a shared React timeline for playback and comparison. StemLab can run as a CLI, Python library, web service, container or local Codex MCP plugin. It is the music-understanding layer of Kieran Simkin's DanceFlow workflow and can supply downstream motion systems such as the WordPress DanceMoves plugin, while remaining useful on its own.

Model weights are not redistributed by this project. They are fetched from their upstream registries/releases on first use. This avoids silently republishing checkpoints whose licensing may differ from the source code license, and lets upstream integrity metadata be used where available.

DanceFlow and DanceMoves

DanceFlow is the wider BPM and motion-response workflow. StemLab is its music-understanding layer; the WordPress DanceMoves plugin is a related downstream motion-response component. StemLab has no WordPress dependency and communicates through normal analysis artifacts and service APIs, so it remains independently useful and reproducible.

See docs/danceflow.md for the component boundary.

StemLab's browser workspace imports the reusable react-timeline-sequence control for audio playback, the shared playhead, seeking, zoom and generic sequence lanes. StemLab retains the upload and analysis-specific adapters.

Curated default model set (September 2026)

An optional htdemucs_6s model is registered for guitar/piano comparison but is not part of the default top-four group.

Relevant upstreams:

• https://github.com/openmirlab/bs-roformer-infer
• https://github.com/ZFTurbo/Music-Source-Separation-Training
• https://github.com/adefossez/demucs
• https://github.com/sigsep/open-unmix-pytorch

Analysis performed

For every successful separator, every WAV it emits is retained. Stem names are discovered from files rather than truncated to a hard-coded four-stem schema, which is important for MVSep Mega 53. For the master and every stem, StemLab writes both a PNG spectrogram and compressed .npz spectrogram data (time axis, frequency axis, dB matrix, FFT metadata).

Full-resolution spectrogram evidence stays in the .npz; the PNG renderer is bounded to 8,192 time frames so long tracks cannot expand into multi-gigabyte Matplotlib RGBA buffers. If an older run retained the NPZ but failed while drawing the PNG, repair it in place without rerunning models:

stemlab repair-spectrograms "path\to\analysis-results"

The preferred vocal stem is passed through the Silero VAD implementation bundled with faster-whisper. Speech regions are used to create a timeline-preserving spoken_word.wav (non-speech is zeroed rather than concatenated), then Whisper is run with word timestamps. Outputs include whisper.json, speech_regions.json, transcript.txt, transcript.srt, and words.tsv.

Whisper runs in repetition-safe mode by default (condition_on_previous_text=False) to prevent a repeated syllable in one window contaminating later windows. whisper.json records that setting and includes repeated-token diagnostics. Use --whisper-condition-on-previous-text only for an explicit comparison run; a flagged transcript remains evidence requiring section-wise recovery or listening QA, not a trustworthy lyric source.

Beat analysis runs:

• BeatNet in offline/DBN mode through StemLab's checksum-verified runtime bootstrap. StemLab uses the maintained madmom-prebuilt wheel and bypasses only BeatNet 1.1.3's obsolete NumPy/Numba package metadata.
• Beat This! using the final0 checkpoint and its minimal postprocessor.
• Beat Transformer, using the original released model code/checkpoints. The model was trained on five demixed mel streams, so StemLab maps BS-RoFormer-SW to vocals, drums, bass, piano, and other + guitar, makes 128-bin mel-power spectrograms at the original 44.1 kHz / 4096 FFT / 1024-hop settings, and averages all eight released fold checkpoints by default. It uses the original madmom DBN decoder if madmom is importable, otherwise a documented SciPy peak-picking fallback.

Beat outputs are stored as JSON, TSV, and (for Beat Transformer) activation NPZ data.

StemLab also supports the official Vamp Plugin Pack, executed through Sonic Annotator. The curated Vamp pass focuses on musically useful outputs: Chordino chord transcription and harmonic-change likelihood; NNLS chroma and bass chroma; Queen Mary key and tonal-change detection; concert-pitch tuning; pYIN melody/F0 and monophonic note transcription on the preferred separated vocal stem; Silvet polyphonic note transcription; Segmentino song-structure segmentation; and the Queen Mary Vamp beat/bar tracker. Raw CSV, pinned transform files, JSON, NPZ and plots are retained under vamp/, with the most useful melody/harmony layers also embedded in the Sonic Visualiser session.

Comprehensive sonic, harmonic, rhythmic and semantic analysis

StemLab 1.0 consolidates a higher-level evidence-fusion pass without discarding any of the existing low-level outputs. The default deep pass now includes:

Optional actions include Basic Pitch polyphonic MIDI/note transcription on isolated instrument stems and MuQ-MuLan zero-shot audio/text semantics. MuQ-MuLan's released weights are CC-BY-NC 4.0, so that route is deliberately opt-in and its licence is embedded in every result. Embedding similarities are labelled as similarities, never probabilities.

Inspect the routes and their dependencies with:

stemlab analysis-actions

Useful switches include --no-structure, --no-text-semantics, --audio-semantics, --basic-pitch, and --all-in-one-embeddings.

The derived artifacts live under deep/ (sonic/, rhythm/, harmony/, structure/, lyrics/, semantic_text/, optional semantic_audio/ and basic_pitch/, plus song_map/song_map.json and summary.json).

Find DanceRudiments matching candidates from each stem

Install the separately maintained optional native package with python -m pip install -e ".[rudiments]". StemLab calls dancerudiments.catalogue() and dancerudiments.sample() directly—no cloned movement functions or bundled catalogue. It extracts per-stem RMS attack peaks, maps them onto detected beats, and ranks 16-beat windows against position/speed/acceleration salience of actual native motion samples. Scores are comparative similarities, not probabilities or guarantees of a good motion.

# Match the mix plus every saved stem without running separation again.
stemlab rudiments ./analysis-master --top 20
# Limit an existing analysis to two stems, or choose one of six official collections.
stemlab rudiments ./analysis-master --stem drums --stem bass --collection club
# Include matching as an opt-in part of a new full analysis.
stemlab analyze master.wav --output ./analysis-master --rudiments
# The bundled Arcadians MP3 has a documented BPM but no pre-separated stems.
stemlab rudiments 'examples/arcadians/Arcadians - 320kbps.mp3' \
  --bpm 145 --collection initial --top 28 --output ./arcadians-rudiments

Measured Arcadians benchmark

The mix-only offline benchmark found 1,821 energy-rise peaks and 15 candidate patterns at ≥0.68 similarity among the authentic 28-pattern DanceRudiments initial collection, led by groove_b_played (0.739), groove_b_grid (0.737), and lfo_morph (0.724). The benchmark read DanceRudiments' officially compiled sample tables through an offline adapter; the native C++-extension test is separate and skips when that optional package is not installed. The BPM grid starts at zero with unverified beat phase. No stem separation was claimed.

These are actual Chromium screenshots of the measured local HTML report, not schematic illustrations or captures of the main React timeline.

The analysis writes deep/rudiments/report.json, index.html, and overview.png. See method, score caveats, provenance, examples and screenshot reproduction.

Native CI coverage: the main GitHub Actions CI workflow now has a dedicated Python 3.10/3.11 job that installs the official DanceRudiments binary wheel, requires the actual C++ module, and runs the Arcadians test with skipping disabled. Fewer than 12 real-audio matches fails CI; passing runs publish the JSON, HTML and measured overlay PNG as downloadable CI artifacts. See the native CI validation instructions.

BPM-synchronised recordings, harness and batch generation

Export any number of independently routed master/stem dimensions at 64 pips per beat, using the separately maintained DanceRudiments float64 recording format. Every exported DanceRudiments dimension is projected onto the same bipolar -1 to +1 movement scale before JSON, legacy-pack or harness output. The normalization option selects the amplitude reference (PCM unity, shared peak, or per-axis peak); it no longer changes the output coordinate range. Silent axes remain centered at 0, and signed gain is applied after bipolar projection.

A BPM estimate or confirmed value is required; raw audio uses --bpm, while saved analysis can supply current beat evidence. --bpm-confirmed records explicit confirmation, separately from estimated/unverified beat phase.

# Bass on X, treble on Y; generate JSON plus the actual DanceRudiments visualizer.
stemlab record-rudiment master.wav --bpm 145 \
  --axis x=master --low-pass x=180 \
  --axis y=master --high-pass y=3000 --output ./bass-treble
# Explore filtered/unfiltered master and every saved stem, including N-D combinations.
stemlab generate-batch ./analysis --bpm 145 \
  --dimensions 2 --dimensions 3 --dimensions 6 \
  --max-candidates 96 --seed 17 --include-audio --output ./movement-batch
# Fourier analysis: 32 frequency buckets become exactly 32 recording dimensions.
stemlab record-rudiment master.wav --bpm 145 --frequency-bands 32 \
  --fft-size 2048 --normalize global --output ./frequency-32
# Reuse a saved stem instead of analysing the master spectrum.
stemlab record-rudiment ./analysis --frequency-bands 32 \
  --frequency-source htdemucs_6s/drums --normalize global --output ./drum-frequency-32

--frequency-bands N maps ascending frequency bands to band_000 through band_N-1, with one periodic-Hann real FFT per pip and window-corrected RMS band levels. The default is equal-width bands from DC to the source Nyquist frequency; --frequency-min, --frequency-max and --frequency-scale linear|log select a different range/spacing. Log spacing requires a positive minimum. --fft-size controls the frequency/time resolution; unresolved empty bands are errors. Channel powers are averaged without cancelling antiphase stereo. The existing clock, bipolar projection, gain, normalization, JSON and native harness routes are reused. This produces band-level motion, not separated audio files or instrument stems. See Fourier options, calibration and boundaries.

Open the output index.html for searchable movement cards, the existing four-card comparison, shared audio transport, review choices/notes, and X/Y/Z selectors for any stored dimension. Native C++/WASM playback and visualizer assets stay in DanceRudiments, not a copied engine or lookalike page in StemLab. Recordings stop at their end, retain all dimensions, and do not silently become loops.

Choose per-axis peak/mean-absolute-amplitude windows and low/high/band-pass routes. Batch mode covers each source, explores seeded combinations, and reuses cached filter results/deduplicated columns instead of an unbounded Cartesian product. Source audio is copied only with --include-audio; no inference or model download runs. Outputs must be new directories. This requires the companion DanceRudiments recording/harness update; older published packages lack the API. Existing matching and XYZ formats remain compatible (--legacy explicitly exports a bounded XYZ loop).

See installation, all options, limits, timing, demo and validation.

Real Chromium capture of the bundled synthetic-audio batch, not Arcadians model inference. Restricted-environment QA used in-memory loading; normal file loading and audible listening are not claimed by that screenshot.

Arcadians: complete analysis timeline and four movement demos

Every new release builds one public site containing five working examples, all covering the full original Arcadians song:

• Analysis timeline: the real StemLab React viewer, original audio, six genuine HTDemucs stems, waveforms/spectrograms, master/percussion beat estimates and consensus, Whisper words, all 13 curated Vamp analyses, functional structure, sonic/rhythm/harmony/lyric analysis, text semantics, song map and loop discovery.

• Movement comparison: the existing 48 DanceRudiments movements with 2D/3D/6D filtered/unfiltered master routes, 145 BPM, 64 pips per beat and float64 recordings. This example remains master-derived; it does not claim its movements use separated stems.

• Stem-pair comparison: 60 two-dimensional recordings with different actual stems driving X and Y.

• Amplitude-mode comparison: 30 fixed input/filter pairs, each with peak, average and four moving-average windows: 180 recordings on shared normalization references.

• Fourier frequency bands: select 4, 8, 16, 32, 64 or 128 dimensions derived from the full original master. Each equal-width band supplies one dimension through a centered Hann 2048-point rFFT and window-corrected band RMS. Global normalization preserves relative band strengths within each resolution; it does not calibrate resolutions against each other. Choose any X/Y/Z axes or see every band in the shared Bars view.

The root landing page links to all five. These are deployment destinations; a patch or a green build alone does not establish that the new page is live.

The timeline's arcadians-core-v2 recipe runs all the selected core families on the complete checksum-verified MP3. It is not the five-separator full comparison profile or every optional MIDI/evidence/NC model. Coverage, dependency versions and source/export revisions are disclosed. Missing stems, beat results, Vamp plugins or required deep-analysis evidence fail the build; there is no synthetic fallback. Artist-confirmed 145 BPM, lyrics, manually edited lyric timing and artist sections stay distinct from machine estimates. A completed loop scan may legitimately return zero accepted loops and explain unresolved sections.

Build and publish all five examples

Start Actions → Arcadians release demo → Run workflow → main. The workflow first runs the full CPU analysis in StemLab's existing Docker environment, then exports and browser-tests the timeline. It also builds/tests the original movement example and both new comparison pages, reusing the exact separated float WAVs, plus all six full-master Fourier resolutions, and combines all five into a single verified Pages artifact. The model analysis is substantially heavier than master-peak extraction; upstream model downloads and inference must succeed. A new tagged release calls the same combined workflow.

All five examples include audio playback, without autoplay. The timeline needs no live StemLab API, Socket.IO connection or CDN: the original React renderer loads precomputed local data, waveform envelopes and spectrogram strips. It supports play/pause, seeking, zoom/Fit, and saved-loop audition when loops exist. The original MP3 is unchanged; public stem audio uses full-duration PCM24 FLAC listening derivatives with their conversion disclosed. Original float WAVs and raw results are kept in a separate seven-day Actions analysis artifact.

Keep Pages Source = GitHub Actions and STEMLAB_PAGES_ENABLED=true for automatic publication after a successful manual demo or a tagged release. Preserve github-pages environment approvals and allow the relevant main/tag refs. Alternatively, Publish Arcadians demo to Pages → Run workflow → main publishes a selected successful combined demo (blank run ID chooses the latest), without rerunning models or making a package release. Old artifacts without all three additional galleries and the timeline are rejected so they cannot overwrite the complete five-example site. The standalone Arcadians complete analysis timeline job is build-only, not a separate Pages deployment.

The publisher verifies all viewers' provenance, artifact checksums and successful HTTP browser reports, then probes the actual public pages, timeline assets/data and partial MP3 responses. Failed or in-memory-only QA cannot authorize publication. QA files, private folders, model caches and runtime logs stay outside the public upload. All five examples are included in StemLab-Arcadians-visualizer.zip, with file hashes and build provenance. Native recording downloads are losslessly compressed as .json.gz; decompress before loading with the native recording API. No samples are discarded or quantized. Every resolution has mandatory native/bar/browser/audio QA. An incomplete build never replaces the live site.

Export an existing analysis as a static timeline

# Install the server extra for shared export helpers; do not start a server.
stemlab export-timeline ./analysis-arcadians \
  --include-audio --output ./arcadians-timeline-site

The output must be a new directory outside the analysis. Host it over static HTTP(S), not file://. Audio/lyric inclusion is explicit. Generic exports show saved evidence without claiming the complete Arcadians recipe ran.

See complete recipe, workflow, static export, timing and validation and movement/audio details and Pages configuration.

Timeline analysis gallery

The browser workspace is intended to make the analysis output inspectable, not just downloadable. The illustrations below explain the main analysis families using the same shared timeline vocabulary as the frontend: one clock, one playhead, zoomable lanes, and consistent section alignment across spectrogram, beat, lyric and deep-analysis layers.

Illustrations, not captured frontend screenshots. These SVGs are generated by scripts/render_readme_analysis_gallery.py; the script does not execute the React timeline component, decode audio, run separation or perform neural inference. Only the Arcadians section labels, lyric text/timing and reference BPM come from the included artist-authored metadata. Other plotted patterns and example chord labels are explicitly schematic, not measured results for Arcadians. In particular, these pictures are not evidence of vocal-free cuts, measured tempo stability, chord/key estimates or loudness values. Actual frontend captures remain outstanding.

1. Separation, spectrograms, beats and sections

This is the core inspection view. It explains how StemLab lines up:

• the AI / reference section map (deep/structure/ and deep/song_map/),
• the beat and bar grid from BeatNet, Beat This! and Beat Transformer (beats/*.json, beats/*.tsv),
• the master spectrogram, and
• each stem spectrogram written under spectrograms/.

In practice, this is the view you use first when checking whether a separator, beat model or section model has produced outputs that are musically believable.

2. Lyrics, speech regions and transcript QA

StemLab keeps lyric and speech analysis on the same clock as the audio:

• reference lyric timing from a known-correct LRC or JSON source,
• Whisper word timing from the preferred vocal stem,
• speech-region / VAD evidence indicating detected speech activity; this alone does not establish the absence of singing,
• the derived spoken_word.wav timeline-preserving speech isolate, and
• downstream rhyme, prosody and repetition-safety diagnostics under speech/ and deep/lyrics/.

The important idea is that you can compare what the model heard against what the artist supplied, then immediately relate both to loop boundaries, choreography cues or lyric-sheet QA.

3. Harmony, melody and song-map fusion

StemLab's harmonic and structural analysis is wider than a single chord track. The harmony/song-map family brings together:

• chord and harmonic-change evidence from the curated Vamp pass,
• key, tonal-change and tuning outputs,
• melody / F0 style lanes derived from vocal analysis,
• the section-level song map that fuses structure, rhythm, harmony and lyric evidence into one readable narrative.

These outputs live primarily under vamp/, deep/harmony/ and deep/song_map/, and they are the layer that makes the raw low-level DSP outputs easier to interpret in musical terms.

4. Sonic profile, rhythm regimes and semantics

The higher-level deep pass also tracks how a song feels over time:

• sonic profile summaries (loudness, peaks, dynamics, timbre and stereo behaviour),
• groove / rhythmic metrics such as swing tendency, offbeat energy and quantisation error,
• tempo-regime detection for stable regions, ramps and abrupt changes,
• semantic summaries from lyric/text analysis and, optionally, zero-shot audio semantics.

This is the analysis family that turns StemLab from a simple BPM-and-stems tool into a richer musical understanding layer for DanceFlow and downstream motion-response systems.

5. Where loops fit in

The loop-discovery patch does not replace the earlier analyses; it depends on them. Loop selection uses:

• section labels to find candidate verse and chorus spans,
• the beat grid to align cuts to complete bars,
• full-vocal stem and available timing evidence to avoid detected vocalisations; VAD alone is insufficient,
• and waveform seam checks to reduce discontinuities; accepted joins still need listening checks.

That means the loop lane is best understood as an applied synthesis of the structure, rhythm and vocal analyses already present in StemLab.

Loop discovery and playback

StemLab now attempts at least one complete-bar loop for every identified verse and chorus occurrence. It checks the beat grid, vocal-clear cut neighborhoods and every audio channel's waveform seam. Unsafe or unsupported sections are reported rather than forced. Start/end sample frames, quality diagnostics and provenance are written to deep/loops/loops.json and loops.tsv by default.

# New analysis: loop metadata is included in the default deep pass.
stemlab analyze master.wav -o analysis-master --profile practical

# Also save exact native-rate WAV slices.
stemlab analyze master.wav -o analysis-master --profile practical --export-loops

# Reuse existing separation, structure and beat results; no models rerun.
stemlab loops analysis-master --export-loops

# Keep all candidates within a chosen short-video edit budget, in seconds.
stemlab loops analysis-master --max-seconds 30 --per-section 3

# Audition possible new cut pairs anywhere in the song, even where the passage
# does not already repeat. These near-misses are NOT accepted seamless loops.
stemlab loops analysis-master --mode exploratory --search-scope whole_song \
  --max-seconds 30 --per-section 20 --exploratory-algorithm spectral_context

In the existing analysis frontend, choose a highlighted loop region or the Loop selector, tick Enable loop, then press Play. The same playhead, seeking, zoom and spectrogram timeline remain in use. Previewing a loop does not require exported WAVs and writes no files. Zoom to loop focuses the shared timeline.

See the complete loop workflow for installation of both patches, sample indexing, thresholds, limitations and reproducible screenshot examples.

For line-complete alternatives when a musically satisfying restart is the goal, use stemlab phrase-loops analysis-master --feedback artist-ratings.json --export-audio. This opt-in search compares the musical context around both cuts, preserves canonical lyric occurrences, uses explicit artist feedback and selects diverse passages. Suggestions remain separate from strict accepted loops. See phrase-loop methodology, feedback format and synthetic example. For longer excerpts, add --min-seconds 20 --max-seconds 30; optional --max-bar-phase-error 0.125 also screens out mismatched measured bar positions. These filters suggest auditions and do not establish seamless or listening approval.

Recover a Short or video's source timing

stemlab match-clip master.wav published-original.mp4 --output new-match.json locates a contiguous soundtrack excerpt without running models or downloading anything. It retains both original hashes, timing precision, competing source positions and independent window checks. Matched reports provide source offset, soundtrack duration and player start for synchronised consumers. Short fades and compression are tolerated; ambiguous repetition, edit jumps, different recordings and rate changes remain unresolved. Use --clip-start / --clip-end for an explicit known soundtrack interval. FFmpeg is required for video decoding. See clip matching for limitations and a synthetic example.

Audio-to-MIDI model comparison

MIDI extraction is an optional, independent model pass. It adds Transkun V2 and high-resolution note/pedal piano transcription, plus MR-MT3 and YourMT3 multi-instrument transcription, alongside Spotify Basic Pitch. Automatic source selection uses piano/keyboard stems for the piano specialists, pitched stems for Basic Pitch and the mix for MT3-family models. Missing stems are reported, never silently substituted. The legacy --basic-pitch command remains supported.

# Install the chosen runtime; the standard profiles do not install MIDI models.
python -m pip install -e ".[midi-transkun]"
stemlab midi-models

# Use an existing separated piano, without rerunning any earlier analysis.
stemlab midi analysis-master --model transkun

# Or transcribe a directly supplied piano recording to a new folder.
stemlab midi piano.wav -o midi-piano --model transkun

# Full mix comparison requires the separately installed midi-mt3 extra.
# Both requested checkpoints need explicit first-use download consent.
stemlab midi mix.wav -o midi-comparison --model mr_mt3 --model yourmt3 --allow-model-downloads

Each model/source gets its original .mid, tempo-aware notes.json and notes.csv, an optional real note piano-roll.png, source/checkpoint hashes, package versions and a worker log. Native MIDI controllers, pitch bends, tempo changes and drums are preserved. Note sample numbers are original-rate estimates, not a claim that the transcription is sample-accurate ground truth. The MIDI clock is not used to overwrite StemLab's measured beat grid.

New analyses can opt in with repeated --midi-model switches and --midi-allow-downloads. Existing Codex installations gain a workspace-scoped stemlab_start_midi_scan job tool after updating the Python installation and restarting the session. Models remain optional and no Python dependencies are automatically installed. Incompatible research runtimes can use separate --backend-python MODEL=PATH interpreters.

See the MIDI guide for all models, installation choices, checkpoint and licensing caveats, saved-stem targeting, Codex usage, exact output semantics, real-inference smoke commands and test limitations. The React timeline remains the analysis inspector; this patch adds no MIDI synthesizer or new interactive piano-roll lane. Static note plots are not frontend screenshots.

Align timed musical cues to detected beats

Use saved analysis to snap an LRC cue file without rerunning models or editing audio:

stemlab snap-cues analysis-master "musical cues.lrc" -o cue-review
stemlab snap-cues analysis-master "musical cues.lrc" -o cue-review-100ms --tolerance-ms 100
stemlab generate-cues analysis-master -o generated-cues --title "Song title"

Only timestamps change: bracketed labels such as [DROP: CHORUS 1], metadata, encoding and line layout are preserved. A cue with no detected beat within ±150 ms is highlighted as REVIEW, with its nearest-beat gap, and kept unchanged by default. --force-snap moves distant cues too but retains their warnings. --downbeats restricts targets to reported bar starts; --beat-model selects a specific detector. No constant-tempo beats are invented to fill detection gaps.

Each new output folder contains the rewritten .beat-snapped.lrc, report.json, a colour-coded review.html, and review.txt. Exact mode preserves the stored beat timestamp; explicit millisecond/centisecond compatibility modes report rounding. Source-rate sample-frame estimates are included when the rate is known. Use --fail-on-review to write results and exit 2 when review is needed. Generated section-cue files use a title-specific, WordPress-safe filename so multiple releases can be uploaded and mapped without ambiguous duplicate names. For the retained 3 October 2026 future-release batch, scripts/run-upcoming-full-cues.ps1 resumes full analyses without overwriting an incomplete directory, and scripts/summarize-upcoming-cues.ps1 reads each closed manifest/report to produce one reproducible hash, cue-action and error-count row per release. The summary script refuses missing or ambiguous generated LRCs.

For a new analysis, add --snap-cues "musical cues.lrc" and optionally --cue-tolerance-ms 100. Cue alignment uses that run's detected events and appears in analysis.json. See the cue alignment guide for exact precision, offsets, all switches, safety rules and interpretation of warnings.

Complementary evidence models

StemLab now has an optional evidence-model layer for models that add a different kind of musical evidence instead of duplicating the normal separation or MIDI backends. Nothing in this layer is enabled by a normal profile.

stemlab evidence-models

# Singing/speech/music activity + continuous pitch on saved vocals.
stemlab evidence analysis-master -o evidence-vocals \
  --model firered_aed --model swift_f0 \
  --model-path firered_aed=/models/FireRedVAD/AED

# Align approved lyrics without replacing them.
stemlab evidence analysis-master -o evidence-align \
  --model qwen_forced_aligner --canonical-text lyrics.txt --allow-model-downloads

# Independent harmony evidence.
stemlab evidence analysis-master -o evidence-chords --model lv_chordia

The built-in registry covers FireRed AED, HeartTranscriptor, Qwen3 ForcedAligner, SwiftF0, SongFormer, lv-chordia and ADTOF PyTorch. It also exposes explicit external-runtime bridges for GAME, SheetSage2, MOSS-Music and AudioSep so large or restricted research dependencies never get silently bundled into StemLab. Outputs stay independent: singing activity, forced alignment, pitch, structure, chords and drum hits are review evidence, not a hidden combined ground-truth score.

Saved analyses automatically route vocal models to a preferred full vocal stem, drum transcription to a drum stem, continuous pitch to pitched stems, and structure/harmony models to the master. Missing specialist stems are reported. Use --backend-python MODEL=/path/to/python for conflicting research environments.

See the evidence-model guide for model terms, routing, external bridges and interpretation rules. scripts/benchmark_evidence_corpus.py probes an authorised evaluation collection without copying audio into the repo.

Install

Python 3.11 is recommended for the broad optional-model ecosystem. The beats and all extras install Beat This! plus the compatible madmom-prebuilt decoder. BeatNet 1.1.3 itself still publishes obsolete NumPy/Numba dependency pins, so StemLab downloads its official pure-Python wheel separately, verifies the published SHA-256, and exposes only the BeatNet code and bundled model weights from its cache. This route is supported on Python 3.11-3.13; a missing BeatNet result must not be mistaken for a successful detector run.

From PyPI, install the released StemLab distribution with:

pip install danceflow-stemlab

The installed Python package and command remain stemlab. For development from a source checkout:

python -m venv .venv
. .venv/bin/activate              # Windows: .venv\Scripts\activate
python -m pip install -U pip
pip install -e ".[all]"

SCNet and Beat Transformer are research repositories rather than stable pip inference APIs. They are isolated/downloaded on demand. To prepare everything in advance:

stemlab bootstrap all

For BeatNet alone:

pip install "danceflow-stemlab[beats]"
stemlab bootstrap beatnet
stemlab doctor

stemlab doctor reports BeatNet as ready only when the normal package or the verified StemLab cache can actually be found. analyze --no-bootstrap never downloads BeatNet and records the unavailable backend instead.

For only the Vamp analysis stack:

stemlab bootstrap vamp

bootstrap vamp downloads the pinned Sonic Annotator runtime and launches the official Vamp Plugin Pack installer. Complete the installer once, then rerun the command if needed to verify that the requested plugin outputs are visible.

CLI

Full requested analysis:

stemlab analyze master.wav --output ./analysis-master --profile full --device auto

Useful alternatives:

# Skip the 53-stem model for a materially lighter run
stemlab analyze master.wav -o ./analysis-master --profile practical

# Run only selected separators
stemlab analyze master.wav -o ./analysis-master \
  --model bs_roformer_sw --model scnet_xl_ihf --model openunmix_umxhq

# Fail immediately rather than recording a backend failure and continuing
stemlab analyze master.wav -o ./analysis-master --strict

# Skip Vamp if the plugin pack is intentionally not installed
stemlab analyze master.wav -o ./analysis-master --no-vamp

stemlab models
stemlab doctor

By default model weights and external research code are cached outside the output folder; all generated analysis artifacts and datasets are written inside the output folder. The copied master is also placed in input/ so the Sonic Visualiser session is portable as one directory.

Output layout

analysis-master/
  input/master.wav
  stems/
    bs_roformer_sw/*.wav
    mvsep_mega53/*.wav
    scnet_xl_ihf/*.wav
    htdemucs_ft/*.wav
    openunmix_umxhq/*.wav
  spectrograms/
    master.png
    master.npz
    <model>/<stem>.png
    <model>/<stem>.npz
    speech/spoken_word.png
    speech/spoken_word.npz
  speech/
    spoken_word.wav
    speech_regions.json
    whisper.json
    transcript.txt
    transcript.srt
    words.tsv
  beats/
    beatnet.json / beatnet.tsv
    beat_this.json / beat_this.tsv
    beat_transformer.json / beat_transformer.tsv
    beat_transformer_activations.npz
  vamp/
    report.json
    transforms/*.n3
    raw/*.csv
    data/*.json
    data/*.npz
    plots/*.png
  deep/
    sonic/
    rhythm/
    harmony/
    structure/
    lyrics/
    semantic_text/
    song_map/
    rudiments/             # optional native DanceRudiments candidate report
      report.json
      index.html
      overview.png
    summary.json
  sonic_visualiser/
    session.sv
    session.xml
    open_sonic_visualiser.bat
    open_sonic_visualiser.sh
  analysis.json
  manifest.json

session.sv is genuine Sonic Visualiser bzip2-compressed session XML. It contains a master pane with beat/downbeat/Whisper time-instant layers and one synchronized waveform + Sonic Visualiser spectrogram pane for every separated stem. The uncompressed session.xml is retained for inspection/debugging. The Windows .bat looks on PATH and in the usual Program Files locations.

Web service

StemLab includes an upload/timeline web UI plus HTTP and Socket.IO APIs for content-addressed analysis jobs. See docs/web.md for launchers, endpoints, events and the Arcadians reference workflow.

Documentation

• DanceFlow workflow and DanceMoves relationship
• Web service
• Containers
• Publishing and releases
• BeatNet compatibility
• Changelog

Codex plugin and MCP server

Inspect through the real React timeline first. The Codex plugin now exposes stemlab_open_timeline to serve saved results in StemLab's bundled react-timeline-sequence frontend. Use an available local browser to compare waveform, spectrogram, beat and section lanes, select/enable loops, zoom and audition the repeat. Exact JSON reports support numerical claims; saved PNGs are a fallback when browser access is unavailable. Screenshots must come from the rendered component, never a redraw. stemlab_close_timeline closes the private, read-only viewer without changing results. See the inspection workflow.

Use StemLab from Codex to analyse timing, inspect real evidence and find/export loopable music sections for Shorts videos. This is a local stdio MCP server plus an installable skill bundle, not a remote audio-upload service or a video editor. It reuses the existing pipeline and preserves prior result folders.

# From this patched checkout, in an activated environment:
python -m pip install -e ".[codex]"
stemlab-codex configure --workspace "/absolute/path/to/music-workspace"
stemlab-codex doctor
codex plugin marketplace add .

Install StemLab from the local plugin directory on a supported Codex host; for CLI-only use, register the MCP command directly. See setup and tools for Windows paths, model extras, consent, polling and exact sample conventions.

Release builds validate the MCP integration and attach a versioned plugin ZIP, checksums and MCP Registry metadata to the GitHub release. A release-only codex-plugins branch provides a marketplace channel after PyPI publication. This does not automatically submit to OpenAI's public Plugins Directory. Distribution and optional MCP Registry publishing explains the separate channels and approval requirements.

Codex skill

The repository includes a reusable Codex skill at skills/use-and-improve-stemlab. It makes StemLab the default analysis engine, selects a proportionate profile, preserves evidence boundaries, and routes reusable capability gaps back into StemLab with tests instead of creating an undocumented parallel stack.

Python API

from pathlib import Path
from stemlab.models import FULL_PROFILE
from stemlab.pipeline import run_pipeline
from stemlab.types import PipelineConfig

result = run_pipeline(PipelineConfig(
    input_wav=Path("master.wav"),
    output_dir=Path("analysis-master"),
    models=FULL_PROFILE,
    device="auto",
))

External runtime/cache details

The BS-RoFormer adapter uses bs-roformer-infer, which manages its own checkpoint registry and SHA-256 verification. SCNet is pinned to the v1.0.15 SCNet XL IHF release asset URLs; StemLab records computed hashes after download. Beat Transformer is cloned from its original upstream and consumes the released fold checkpoints from that repository. Set STEMLAB_CACHE=/some/path to relocate StemLab's external cache.

The full run is intentionally expensive. Mega-53 is the dominant VRAM/storage pass and dozens of stems mean dozens of additional spectrogram files and Sonic Visualiser panes. Use --profile practical while iterating and --profile full for exhaustive analysis.

Releases

Version tags publish StemLab to GitHub Releases, PyPI (danceflow-stemlab), GHCR and Docker Hub after validation. See docs/publishing.md for the release contract and Trusted Publishing setup.

Reproducibility and licensing

analysis.json records model/backend metadata and errors; manifest.json hashes every output artifact. Model checkpoints, research code and datasets retain their upstream terms. In particular, do not assume that an MIT-licensed inference wrapper automatically grants redistribution rights for every checkpoint it can download.

Arcadians timeline CI decoder and artifact checks

The complete Arcadians demo checks lint before expensive analysis. Its Docker runtime includes pinned TorchCodec and performs a synthetic MP3 decoder smoke test; the timeline builder also checks the original Arcadians MP3 before model inference. Missing decoders fail early without changing the source audio or relaxing the complete-analysis/publication gates. After Docker exits, the workflow restores runner ownership of its dedicated output folder so private-mode JSON files remain readable for browser validation and diagnostic artifact upload, including on failed analysis runs. See Docker checks and the timeline workflow.

For a new combined timeline and movement deployment, run Arcadians release demo on main. It calls Arcadians complete analysis timeline internally; starting that standalone workflow too duplicates analysis and does not itself publish the combined Pages site.

The Docker build also checks audio-first lyric-semantic imports in a fresh, offline process: it exercises both resampy Kaiser filters before resolving the actual Transformers and SentenceTransformer classes. This catches lazy-loading failures before full-song inference. The container runtime temporarily pins resampy==0.4.2 and setuptools==81.0.0 because Basic Pitch 0.4.0 excludes the newer resampy that no longer needs pkg_resources. These are container-only compatibility pins, not a downgrade of normal package-build tooling. This smoke test does not download weights or claim semantic inference; the full analysis and browser/publication gates remain required.

Separated-stem movement batches and controlled smoothing

stemlab generate-stem-batch separates a raw song once (default htdemucs_6s) or reuses a saved analysis, then generates 2D DanceRudiments with different stems on X and Y. It covers raw ordered pairs and seeded low-/high-/band-pass variants; 120 candidates is the bounded default. Supply a BPM estimate or confirmed value for raw audio. --analysis-output keeps the prepared stems for reuse.

stemlab generate-stem-batch song.wav --bpm 145 --bpm-confirmed \
  --analysis-output ./song-stems --include-audio --output ./stem-pairs
stemlab generate-average-comparison ./song-stems --bpm 145 --max-pairs 32 \
  --window-beats .125 --window-beats .25 --window-beats .5 --window-beats 1 \
  --include-audio --output ./average-comparisons

The second command keeps each pair's source, filters, orientation and amplitude scale fixed while comparing peak, per-pip average, and moving-average settings. Moving average is a separate centered smoothing window on the rectified amplitude envelope, configurable in beats; it does not average signed audio or change the 64-pips-per-beat / float64 recording format. It is also available in record-rudiment and generate-batch via --mode moving-average and --moving-average-beats; individual axes use --axis-window-beats NAME=BEATS.

The Arcadians workflow now assembles five examples: the existing arcadians/ movements and timeline/arcadians/ analysis, plus arcadians-stems/ (60 separated-stem movements) and arcadians-averages/ (30 pairs × six settings). It hands off the timeline analysis's real float stems, without rerunning separation, and bundles the original song for synchronized playback. Both new galleries have mandatory native/browser/audio checks and public-URL verification before success is reported. The Fourier page adds six full-master resolutions without rerunning separation. All pages remain in one Pages deployment. See settings, boundaries, budgets and examples.

Beat detection after separation: master and percussion hypotheses

With beat analysis enabled, the pipeline completes separation first, then runs BeatNet and BeatThis on the master and every labelled drum/percussion stem from every successful selected separator. Each result retains its own source hash, separator, role and timeline lane. Master JSON filenames remain compatible; percussion files have distinct source-qualified IDs. Failed sources are recorded separately, and no guessed percussion stem is substituted.

All-In-One structure remains master-based, and Beat Transformer runs once on its actual five demixed streams—not repeatedly on filenames it does not consume. Consensus allows one vote per detector family, not one per stem, preserving all source hypotheses without letting repeated drum runs dominate the vote. The arcadians-core-v2 publication gate requires the new, correctly attributed master/drum beat results. See routing, provenance and failure handling.

Timeline overlays and movement display modes

Master/stem audio appears as one channel per source, with Waveform / Spectrogram / Both in the left heading. Both overlays the waveform on the time-aligned spectrum. Each channel has a height slider and reset: wheel pans time, Ctrl/Command + wheel zooms time, Alt + wheel resizes the hovered channel, and Shift + wheel scrolls between channels. These are display-only changes.

The shared DanceRudiments Movement view selector switches between the existing X–Y trajectory and Bar graph, one labelled signed bar per stored dimension. It retains the original native sampler, audio clock and review state. BeatNet/BeatThis still analyse both the master and every drums/percussion stem after separation; the master pass is never replaced.

See viewer controls and the released-viewer workflow. Release both companion packages first; StemLab resolves the latest stable official npm/PyPI releases and rebuilds from those distributions. Development pins and local vendor archives are not production dependencies. Existing exported pages must be regenerated.

Coordinated releases and latest-release dependencies

Keep DanceRudiments, react-timeline-sequence, and stemlab as sibling checkouts. The release order is React Timeline Sequence → DanceRudiments → StemLab: publish and verify the companions before updating the consuming app. StemLab must use their latest stable official releases, not an unreleased commit, sibling source package or vendored npm archive. Each build records exact released versions/commits/integrity after latest-release resolution so all its jobs use the same bytes; CI fails clearly when that snapshot is stale.

From the parent folder, preview and then explicitly execute:

.\stemlab\scripts\Release-DanceFlow.ps1 -ProjectsRoot .
.\stemlab\scripts\Release-DanceFlow.ps1 -ProjectsRoot . -Execute

For the first migration use the same commands with .\release-kit\Release-DanceFlow.ps1 after extracting the complete kit. It applies the three repository patches, updates versions, performs scoped commits and normal pushes, waits for exact-commit CI, then creates new release tags. StemLab's CLI, both plugin manifests and MCP registry metadata are versioned together. The existing GitHub Actions workflows publish packages; no protection rules, credentials, old tags or assets are changed. The initial expected next-minor versions are only a preview proposal, never a claim that they are published.

Use Node.js 24, Python 3.11+, PowerShell 5.1+, Git and authenticated GitHub CLI. Required main/registry/Pages jobs must pass; missing approvals and partial publication are blockers, not success. The default script releases all three projects, is opt-in via -Execute, and supports a saved exact-source -Resume session. A changed checkout or pre-commit build failure requires review rather than an automatic stash/reset or guessed recovery.

StemLab's helper updates npm manifests/lock, optional native dependency minimum, release/build metadata and the genuinely rebuilt JS/CSS together. Its daily release-update workflow proposes subsequent updates by PR without auto-merging. A tagged StemLab release generates all five Arcadians pages and validates actual Pages deployment, live HTML and seekable audio. Do not start a duplicate manual Arcadians analysis during that tagged build. Read the full release, resume, verification and troubleshooting guide before running it. Keep this README and AGENTS.md synchronized with future changes.

Standalone Linux CLI release gate

The Linux one-file stemlab-linux-x86_64 executable is a required release artifact, not a best-effort extra. It is packaged with PyInstaller, so normal Python/native modules are bundled rather than translating the complete ML stack into C. Its dedicated Standalone CLI workflow uses Python 3.11 so the maintained madmom-prebuilt decoder and StemLab's checksum-verified BeatNet 1.1.3 runtime can coexist. The build bootstraps the official BeatNet wheel, verifies its SHA-256, embeds the BeatNet code and model weights plus required package data/metadata, then smoke-tests the actual executable with an empty external StemLab cache. --help, models, and doctor must work, and doctor must report embedded BeatNet as ready.

Main-branch release coordination requires binary.yml to pass before tagging. Tagged releases call the same reusable workflow and cannot publish a fallback STEMLAB-BINARY-NOTE.txt; absence or failure of the executable fails the release.