eluate 0.0.3

Remove background music from videos - for accessibility and personal use

ELUATE

ELUATE is a command-line tool that removes the background music from a video and keeps the dialogue and sound effects. It's for people who want to watch documentaries, lectures, and other long-form video without a score running underneath: for religious reasons, or because hearing loss or an auditory-processing difference makes the music fight the narration.

I'm Muslim, and I kept hitting the same wall. The documentary I wanted to watch almost always had a score under it, and my usual move was to just skip it. So I built the thing I wanted: hand it a video, get the same video back with the music gone. The video stream is copied through untouched, so the picture stays bit-for-bit identical and only the audio changes.

eluate documentary.mp4
# → ~/Documents/ELUATE/documentary_eluted.mp4

There's a small Python API behind the same engine, so you can wire ELUATE into a batch job or another tool:

import eluate

eluate.elute("documentary.mp4")

Install

pip install eluate

ELUATE needs FFmpeg on your PATH. Install it with brew install ffmpeg on macOS or sudo apt-get install -y ffmpeg on Debian/Ubuntu. It's already there on Colab.

The first time you run eluate, it downloads a ~450 MB model checkpoint from Zenodo into ~/.eluate/models/. That happens once. To download it up front (and to make the Python API usable, since the API never downloads on its own), run:

eluate setup
eluate info     # check device, FFmpeg, and model status

On a machine with no CUDA or MPS, pip install 'eluate[cpu]' pins CPU-only PyTorch wheels and saves about a gigabyte.

A ready-to-run Colab notebook is at notebooks/eluate_colab_template.ipynb. Set the runtime to GPU (Runtime -> Change runtime type -> GPU) and run the cells top to bottom.

Usage

eluate                                 # interactive: prompts for a file path
eluate video.mp4                       # single file → ~/Documents/ELUATE/
eluate video.mp4 -o cleaned.mp4        # custom output path
eluate --folder ./videos               # every video in a folder
eluate --folder ./videos --batch-size 10   # folder, pausing every 10 files
eluate --batch files.txt               # paths listed one per line in a file
eluate --checkpoint eng video.mp4      # English-tuned model instead of multilingual
eluate video.mp4 --device cpu          # force CPU (skip MPS/CUDA)
eluate video.mp4 --force               # skip the duration and disk-space prechecks
eluate video.mp4 --audio-codec alac    # write lossless audio instead of AAC

The folder and interactive modes accept these containers: mp4, mkv, avi, mov, webm, flv, wmv, m4v, mpeg, mpg, 3gp, 3g2, ts, mts, m2ts. Passing a file directly hands it to FFmpeg regardless of extension, so anything FFmpeg can demux will work.

Output audio defaults to AAC at 256k (--audio-codec, --audio-bitrate). Inference picks the best device automatically (CUDA, then MPS, then CPU); --device overrides it.

Languages

The model ships per-language checkpoints: multi (default), eng, deu, fra, spa, cmn, fao. Switch with --checkpoint eng. Each one is a separate ~450 MB download fetched on first use.

Python API

For more than one file, use a Session so the model loads once and is reused across calls:

import eluate

with eluate.Session() as session:
    for path in ["a.mp4", "b.mp4", "c.mp4"]:
        session.elute(path)

eluate.elute(), eluate.Session, eluate.Result, and the eluate.EluateError exception hierarchy are the whole public surface. The API can also return the separated speech and sfx as WAV files for downstream tools (Whisper, loudness analysis, etc.) without re-extracting the audio:

result = eluate.elute("interview.mp4", outputs=("speech", "sfx"))
# result.speech, result.sfx → WAV paths; result.video → None (not requested)

Full reference, including progress callbacks and the typed exceptions, is in docs/api.md.

How it works

ELUATE extracts the audio with FFmpeg, runs it through the Bandit v2 separator to split it into speech, music, and sfx, drops the music stem, mixes speech and sfx back together, and muxes that new audio track onto a straight copy of the original video.

 input.mp4 ─┬─▶ ffmpeg extract ─▶ 48 kHz WAV ─▶ Bandit v2
            │                                      │
            │                    ┌─────────────────┼─────────────────┐
            │                    ▼                 ▼                  ▼
            │                  speech            music               sfx
            │                    │               (drop)              │
            │                    └────────────── mix ────────────────┘
            │                                     │
            └─▶ ffmpeg mux ◀───────  speech + sfx audio track
                   │
                   ▼
           output_eluted.mp4   (video stream copied as-is)

The separator streams the audio through a fixed-size ring buffer instead of loading the whole track into memory, so peak memory stays roughly flat no matter how long the file is. On an 84-minute documentary on an M4 Mac with 24 GB, peak memory was about 19 GB versus about 41 GB for the unoptimized upstream reference (which then swapped ~20 GB to disk). Methodology and raw numbers are in docs/bench/.

Known limitations

• It does well on clean mixes, however when music and speech or music and rythmic sfx (like train sound) overlap heavily you can get faint musical residue or a slight change to the voice. It's better to listen the result before relying on it.
• It is not faster than Demucs. I tuned the memory path for long files, not raw throughput, so on short clips Demucs will finish first. (Demucs is also aimed at music stem separation, which is a different job.)
• The CLI is video-in, video-out and never gives you stems. The Python API can export the speech and sfx WAVs, but neither surface ever exposes the music stem. ELUATE removes music; it does not hand it to you.
• There's a 6-hour duration cap and a disk-space precheck. Both are there to catch misdirected inputs; pass --force to override either.
• Only the default multi checkpoint has a recorded SHA-256. The other language checkpoints download without an integrity check and print a note when they do.
• I developed and tested this mostly on a Mac mini (Apple Silicon / MPS), with the CUDA path tuned for Colab. macOS and Linux are supported; Windows isn't tested.

Files and privacy

Everything lives under your home directory:

Path	What's there
~/.eluate/models/	Downloaded checkpoints (~450 MB each) and config
~/Documents/ELUATE/	Processed output videos
~/.eluate/telemetry.jsonl	Local debug log, only if you enable it

Nothing is sent anywhere after the one-time model download. The debug log is off by default; set ELUATE_TELEMETRY=1 to write a local JSONL record of processing stages, which helps when reporting a bug. It stays on your machine and contains nothing that identifies you or your files.

License and credits

ELUATE's own code is MIT (see LICENSE). The Bandit v2 model weights are CC-BY-SA 4.0, from Zenodo 12701995. CC-BY-SA permits commercial use, but under share-alike terms that are operationally hostile to a lot of commercial software, so talk to a lawyer before shipping a commercial product built on ELUATE. This README is not legal advice.

Credits:

• Karn N. Watcharasupat, Chih-Wei Wu, and Iroro Orife for the Bandit v2 architecture and the Divide-and-Remaster v3 dataset.
• ZFTurbo for the Music-Source-Separation-Training framework (MIT) that ELUATE's separator is built on.
• Built on PyTorch and torchaudio, librosa, Rich for the terminal UI, and FFmpeg.

It's a one-maintainer project plus AI. Fork it, improve it, open an issue or PR at https://github.com/anaxoniclabs/ELUATE/issues.