libspeech 0.2.0

LibSpeech

Lightweight Speech Processing Library

C++ library with Python bindings for audio I/O, DSP, and ONNX-based speech models -- built to install with nothing but pip, no system dependencies (libcurl, OpenSSL, ...) required.

✨ Key Features

• 🚀 ONNX Runtime powered models, without a heavy LibTorch dependency
• 🎙️ Audio I/O (speech::io): load/save WAV/MP3/FLAC, playback, resample, mono-mixing
• 🧮 DSP (speech::dsp): Resample, window functions, FFT/IFFT, DCT/IDCT, STFT/ISTFT, MFCC -- all with their own C++ unit tests and Python bindings
• 🔊 Speech models (speech::models): a Denoiser interface (Facebook/SpeechBrain backends) and Silero VAD
• 🖥️ Cross-platform: Windows, Linux, macOS
• 🐍 Python bindings (nanobind) mirroring the C++ API 1:1
• 📦 Zero system dependencies: no apt install libcurl-dev / OpenSSL needed -- HTTPS model downloads go through a vendored cpp-httplib + Mbed TLS built from source

📦 Installation

Python package

pip install libspeech

From source (C++/CMake)

git clone https://github.com/MohammadRaziei/libspeech.git
cd libspeech
git submodule update --init src/third_party/miniaudio src/third_party/dr_libs src/third_party/indicators

# Mbed TLS is pinned to v3.6.2 and kept as a shallow submodule (see .gitmodules):
git submodule update --init src/third_party/mbedtls
git submodule update --init --depth 1 src/third_party/mbedtls/framework

mkdir build && cd build
cmake ..                      # downloads ONNX Runtime automatically on first configure
cmake --build . -j$(nproc)
ctest                         # runs the full test suite (see "Testing" below)

Only want the DSP layer (no ONNX Runtime/network access needed at all)?

cmake .. -DBUILD_MODELS=OFF
cmake --build . -j$(nproc)

🚀 Quick Start

Python

import libspeech

# --- Audio I/O ---
audio = libspeech.Audio()
audio.load("sample.wav")
mono = audio.to_mono()
resampled = mono.resample(16000)
resampled.save("sample_16k_mono.wav")

# --- DSP ---
mfcc = libspeech.MFCC(libspeech.MFCCParams())
coefficients = mfcc.compute(resampled.data(0))   # [num_frames][num_coefficients]

# --- Speech models ---
denoiser = libspeech.Denoiser.create("facebook", "facebook_denoiser.onnx")
clean = denoiser.process(resampled.data(0))

vad = libspeech.SileroVad()
vad.process(resampled.data(0))
for segment in vad.get_speech_timestamps():
    print(f"speech from {segment.start_s:.2f}s to {segment.end_s:.2f}s")

C++

#include "libspeech/audio.h"
#include "libspeech/dsp/mfcc.h"
#include "libspeech/models/denoiser.h"

speech::io::Audio audio;
audio.load("sample.wav");
auto resampled = audio.to_mono().resample(16000);

speech::dsp::MFCC::Params params;
params.sampleRate = 16000;
speech::dsp::MFCC mfcc(params);
auto coefficients = mfcc.compute(resampled.data(0));

auto denoiser = speech::models::Denoiser::Create("facebook", "facebook_denoiser.onnx");
auto clean = denoiser->process(resampled.data(0));

See examples/ for complete, runnable programs.

🧪 Testing

Tests are organized by language and module (speech_test_<language>_<module>), all discoverable via cmake --build build --target help:

speech_test                       # everything
├── speech_test_cpp
│   ├── speech_test_cpp_dsp       # speech::dsp (C++), no network/ONNX needed
│   └── speech_test_cpp_models    # speech::models (C++)
└── speech_test_python
    ├── speech_test_python_audio  # libspeech.Audio
    ├── speech_test_python_dsp    # libspeech.Resample/FFT/STFT/MFCC/...
    └── speech_test_python_models # libspeech.Denoiser/SileroVad

Run everything with cmake --build build --target speech_test, or just ctest for the same suites without the extra build-tool chatter.

🔧 Architecture

libspeech is split into three independent C++ libraries (each with its own CMake target/namespace/Python module), aggregated by an umbrella speech library:

See checklist.md for the detailed, up-to-date state of the project (what's done, what's in progress, known issues), and audioflux_issues.md for a couple of real bugs found in AudioFlux while vendoring it (with repro steps and fixes, in case they're useful upstream).

📊 Benchmarking

benchmarks/ is a fully standalone CMake project (no relationship to the repository root) that fetches libspeech from GitHub via FetchContent, exactly like it would fetch any competitor library -- libspeech is never a special case in there. Mirrors the layout of tests/: one folder per language.

cd benchmarks
cmake -S . -B build-bench -DCMAKE_BUILD_TYPE=Release
cmake --build build-bench --target libspeech_benchmarks

benchmarks/cpp/bench_stft.cpp measures speech::dsp::STFT, specifically to A/B test whether enabling OpenMP's parallel-frame path (LIBSPEECH_ENABLE_OPENMP, on by default) actually helps -- STFT frames are independent so this should scale with core count, but this couldn't be verified in this project's own single-core development sandbox (see checklist.md). If you have a multi-core machine, please run this and report back:

cmake -S . -B build-bench -DCMAKE_BUILD_TYPE=Release -DLIBSPEECH_ENABLE_OPENMP=ON
cmake --build build-bench --target bench_stft && ./build-bench/cpp/bench_stft

cmake -S . -B build-bench -DLIBSPEECH_ENABLE_OPENMP=OFF
cmake --build build-bench --target bench_stft && ./build-bench/cpp/bench_stft

Compare the two avg=...ms numbers -- that's the real effect of enabling OpenMP on your hardware.

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.