ensemble-pitch-extractor 0.1.1

FCPE TTA and pYIN ensemble pitch extraction for singing voice.

中文|English

Ensemble Pitch Extractor

Ensemble Pitch Extractor is a singing-voice F0 extractor that combines FCPE test-time augmentation with a pYIN high-frequency candidate in a dynamic programming decoder. It is designed for ordinary singing, high notes, and whistle-register material where a single extractor may fail.

The package provides:

• a Python API for extracting F0 from waveforms or audio files;
• a command-line interface that saves .npy F0 tracks and optional .png diagnostic plots;
• an ensemble decoder that selects a smooth candidate path instead of averaging incompatible F0 estimates.

Demonstration

Audio samples sourced from the internet.

Installation

pip install ensemble-pitch-extractor

For local development:

uv sync
uv run ensemble-pitch-extractor --help

Python 3.12 or newer is required. To enable diagnostic plots, install matplotlib separately:

pip install matplotlib

See examples/plotting.py for the plotting functions.

Command Line

Extract F0 from one audio file:

ensemble-pitch-extractor input.wav -o f0_out

This writes:

f0_out/input.npy

The .npy file is a one-dimensional float32 array in Hz. Unvoiced frames are stored as 0.

Save a plot of F0 overlaid on a mel spectrogram:

ensemble-pitch-extractor input.wav -o f0_out --plot

Process all supported audio files in a directory:

ensemble-pitch-extractor audio_dir -o f0_out --plot

CUDA is auto-detected by default. To force a specific device:

ensemble-pitch-extractor audio_dir -o f0_out --device cpu

Control GPU memory with --max-batch-length (default 480000 samples ≈ 30s):

ensemble-pitch-extractor audio_dir -o f0_out --max-batch-length 200000

Disable pYIN and use FCPE TTA only:

ensemble-pitch-extractor input.wav -o f0_out --no-pyin

Useful options:

--f0-min 80
--f0-max 4000
--max-batch-length 480000
--device auto
--pyin-priority-min-f0 1300
--pyin-fcpe-close-semitones 1.0
--interp-uv
--recursive

Python API

from ensemble_pitch_extractor import extract_f0_from_file, load_model

model = load_model()  # auto-detects CUDA, or pass device="cpu"
f0, times, *_ = extract_f0_from_file(
    "input.wav", model=model, f0_min=80, f0_max=4000,
)
# f0: Hz, shape (frames,)   times: seconds, shape (frames,)

For audio already in memory:

import librosa
from ensemble_pitch_extractor import extract_f0, load_model

model = load_model()
sr = model.get_model_sr()
audio, _ = librosa.load("input.wav", sr=sr, mono=True)
f0 = extract_f0(audio, sr, model, f0_min=80, f0_max=4000)

For torch tensor input (padded batch or concatenated, supports GPU):

import torch
from ensemble_pitch_extractor import extract_f0_from_tensor, load_model

model = load_model("cuda")

# padded batch: (batch=4, samples) with fixed-length clips
wav = torch.randn(4, 16000, device="cuda")
f0 = extract_f0_from_tensor(wav, sr=16000, model=model)  # (4, frames)

# concatenated: clips of different lengths, no padding waste
wavs = [torch.randn(8000, device="cuda"), torch.randn(12000, device="cuda")]
lengths = [len(w) for w in wavs]
concat = torch.cat(wavs)
f0 = extract_f0_from_tensor(concat, sr=16000, model=model, lengths=lengths,
                            max_batch_length=20000)  # (2, max_frames), NaN padded

Method Overview

The decoder treats each extractor output as a candidate trajectory. Current candidates are:

FCPE key shift = 0
FCPE key shift = -12
FCPE key shift = +12
pYIN

For an FCPE candidate with key shift $s$, the model output is mapped back to the original pitch space before fusion:

$$ \hat f_{t,s} = \frac{f_{t,s}}{2^{s/12}} . $$

pYIN is included as an ultra-high frequency candidate. By default it only searches 1300–4000 Hz, and frames below 1300 Hz are discarded. This prevents pYIN from replacing FCPE in normal ranges where FCPE usually captures finer detail.

All candidates are converted to MIDI note space:

$$ n_{t,k}=69+12\log_2\frac{f_{t,k}}{440}. $$

The final path is selected by dynamic programming:

$$ \pi^*=\arg\min_\pi \sum_t U_t(\pi_t)+\sum_{t=1}^{T-1} C_t(\pi_{t-1},\pi_t). $$

Here $U_t(k)$ is a per-frame candidate prior and $C_t(i,k)$ is a transition cost. This formulation avoids averaging octave errors, half-frequency errors, and algorithm-specific mistakes into spurious intermediate pitches.

Heuristics as Priors

The implementation uses the following structured priors:

• MIDI-space costs make equal musical intervals comparable across frequency ranges.
• UV penalty discourages fragmented voiced/unvoiced paths.
• Octave-aware jump cost allows one-, two-, and three-octave transitions, which are important for chest-to-whistle jumps.
• FCPE +12 receives a low-pitch prior below E2.
• FCPE -12 receives a high-pitch prior above D5.
• pYIN receives a high-frequency prior only when it is above 1300 Hz and more than one semitone away from every FCPE candidate.
• RMS energy gating removes false voiced output during silence after decoding.

The default candidate order is FCPE 0, FCPE -12, FCPE +12, pYIN, so that exact ties prefer FCPE over pYIN.

Build and Publish

uv lock --python 3.12
uv build
uv publish

With a PyPI token:

uv publish --token "pypi-..."