altasr 1

ALTASR: a scalable, streaming-ready Conformer-CTC speech recognition toolkit, built for Kinyarwanda and reusable for any language.

ALTASR

Scalable, streaming-ready speech recognition — built for Kinyarwanda, reusable for any language.

ALTASR is a Conformer-CTC automatic speech recognition (ASR) toolkit by the ALTA Project (YaliLabs). It covers the full lifecycle — data preparation, training, fine-tuning, evaluation, benchmarking, streaming inference, and edge deployment — behind a small, consistent API and a set of one-line CLIs.

pip install altasr

That single command installs everything needed for training and inference (PyTorch, torchaudio, audio codecs). Optional features are extras:

pip install altasr[bpe]      # subword (BPE) tokenizer
pip install altasr[onnx]     # ONNX export for edge / CPU runtimes
pip install altasr[whisper]  # Whisper baselines for benchmarking
pip install altasr[mic]      # live microphone streaming
pip install altasr[all]      # everything

---

Full reference (dataset format, every CLI parameter, multi-GPU, web integration): docs/USAGE.md

Highlights

• Modern Conformer-CTC encoder with relative positional attention, intermediate CTC regularization, and stochastic depth — accuracy keeps improving as you scale data and model size.
• True streaming: chunked-attention models plus a StreamingSession API and altasr-stream CLI for live captioning from files or microphone.
• Punctuation-aware: the tokenizer and normalization keep punctuation, so transcripts come out readable (ndashaka amazi, urakoze.).
• Smart on your domain, automatically: training builds an n-gram LM
  • word lexicon from your transcripts and ships them in the checkpoint; beam decoding uses them so corpus terms ("DGX", names, places) come out right with no hardcoded lists. Hotwords remain for never-seen terms.
• Your choice of decoder: decoder="auto"|"greedy"|"beam" everywhere — auto picks the best default (beam for single files, greedy for bulk and streaming); explicit choices always win.
• LLM & agent ready: structured results (confidence + word timestamps), an OpenAI tool schema so agents can call ASR as a tool, and optional LLM post-correction through any OpenAI-compatible endpoint (SmartTranscriber runs the whole pipeline).
• Any language: build a tokenizer (char or BPE) directly from your dataset; nothing is Kinyarwanda-specific except the released checkpoints.
• Robust to imperfect data: speed perturbation, SpecAugment, and capacity-scaled regularization are on by default; unreadable/corrupt files are skipped with a warning instead of crashing a training run.
• Efficient everywhere: int8 dynamic quantization for CPU, ONNX export for edge devices, bucketed batching and AMP for GPU training.
• Fine-tuning pipeline: adapt a trained checkpoint to a new dataset (or a new language) in one command, with tokenizer extension and layer freezing.
• Professional benchmarking: compare your checkpoints against Whisper and generate a Markdown/HTML report with WER, CER, error breakdowns and RTF.

---

Quickstart: transcribe an audio file

from altasr import Transcriber

asr = Transcriber.from_pretrained("path/to/checkpoint")
text = asr.transcribe("recording.mp3")
print(text)

With error handling for real applications:

from pathlib import Path
from altasr import Transcriber

def safe_transcribe(checkpoint: str, audio_path: str) -> str | None:
    try:
        asr = Transcriber.from_pretrained(checkpoint)
    except FileNotFoundError:
        print(f"Checkpoint not found: {checkpoint}")
        return None
    except ImportError as e:
        print(f"Missing dependency: {e}")   # e.g. torch not installed
        return None

    if not Path(audio_path).exists():
        print(f"Audio file not found: {audio_path}")
        return None
    try:
        return asr.transcribe(audio_path)
    except Exception as e:                  # unreadable / corrupt audio
        print(f"Could not transcribe {audio_path}: {e}")
        return None

Command line:

altasr-transcribe --checkpoint runs/my_model --audio recording.mp3

Scenario: get domain terms right (hotwords) + structured output

text = asr.transcribe("meeting.mp3")                     # auto = beam + learned LM/lexicon
text = asr.transcribe("meeting.mp3", decoder="greedy")   # fastest, explicit
text = asr.transcribe("meeting.mp3",                     # + hotwords for terms
                      hotwords=["NewClientName"])        #   not in training data

res = asr.transcribe_detailed("meeting.mp3")
res.decoder                       # e.g. "beam+lm+lexfix" — shows what fired
res.text, res.confidence          # "twaguze DGX nshya.", 0.94
res.words[0]                      # Word(word='twaguze', start=0.12, end=0.58, confidence=0.97)

altasr-transcribe --checkpoint CKPT --audio meeting.mp3 --hotwords "DGX,H200" --detailed

Scenario: LLM pipelines and agents

from altasr import Transcriber, LLMCorrector, SmartTranscriber

asr = Transcriber.from_pretrained("runs/my_model/best")
smart = SmartTranscriber(
    asr, hotwords=["DGX", "YaliLabs"], beam_size=8,
    corrector=LLMCorrector(base_url="http://localhost:11434/v1",
                           model="llama3.1"))       # any OpenAI-compatible API
result = smart.transcribe("meeting.mp3")             # dict: text, asr_text,
                                                     # confidence, words, ...

Let an LLM agent call ASR as a tool (OpenAI function calling):

from altasr import ASR_TOOL_SCHEMA, handle_tool_call
resp = client.chat.completions.create(model=..., messages=msgs,
                                      tools=[ASR_TOOL_SCHEMA])
result = handle_tool_call(asr, resp.choices[0].message.tool_calls[0].function.arguments)

Full agent loop, design notes, and streaming-agent guidance: docs/USAGE.md.

Scenario: CPU / edge deployment

Quantize to int8 at load time (CPU only, ~4x smaller Linear layers, faster):

asr = Transcriber.from_pretrained("runs/my_model", device="cpu", quantize="int8")
print(asr.transcribe("recording.wav"))

Or export to ONNX for onnxruntime / mobile:

pip install altasr[onnx]
altasr-export runs/my_model --format onnx --out deploy/model.onnx

The exporter writes model.onnx, a model.json metadata sidecar (sample rate, mel settings, vocabulary), and copies tokenizer.json next to it, so the deployment folder is self-contained.

Scenario: live captioning (streaming)

Train or download a streaming preset checkpoint, then:

# From a file, paced in real time (add --fast to run as fast as possible)
altasr-stream --checkpoint runs/streaming_model --audio talk.wav

# From the microphone (pip install altasr[mic])
altasr-stream --checkpoint runs/streaming_model --mic --quantize int8

In Python:

from altasr import Transcriber

asr = Transcriber.from_pretrained("runs/streaming_model")
session = asr.stream(chunk_seconds=0.6)

for chunk in audio_chunks:            # your capture loop: 1-D float32 @ 16 kHz
    new_text = session.push(chunk)
    if new_text:
        print(new_text, end="", flush=True)
print(session.finish())

Offline (non-streaming) checkpoints also work with altasr-stream, but a streaming-preset model is trained with chunked attention and causal convolutions, so its live accuracy is much closer to its offline accuracy.

Scenario: train on your own dataset (any language)

Metadata is JSON/JSONL/CSV/TSV with an audio path and a transcript per row.

altasr-train \
    --preset medium \
    --audio-root /data/my_corpus \
    --train /data/my_corpus/train.json \
    --val   /data/my_corpus/dev.json \
    --tokenizer bpe --bpe-vocab-size 1024 \
    --out-dir runs/my_model

• The tokenizer (char by default, BPE with --tokenizer bpe) is built from your training transcripts, so any language works out of the box.
• Punctuation is kept by default; pass --no-punctuation for bare text.
• Presets: small (~7M params), medium (~23M), large (~118M), streaming (medium-sized, chunked attention for live use).
• Training auto-resumes from the last checkpoint in --out-dir.

Dataset format — metadata is any of: a JSON list of {"audio_path": ..., "text": ...} objects, a JSON dict keyed by utterance id, JSON-lines, or CSV with a header. Common field names are auto-detected (audio_path/path/file/audio for audio; text/sentence/ transcription/transcript for the transcript; optional duration). Audio can be wav/mp3/flac/ogg/m4a/… at any sample rate — ALTASR resamples internally. Validate your dataset in seconds with --dry-run. Full details and a worked formatting example: docs/USAGE.md.

Multi-GPU — same command, launched with torchrun (one process per GPU; gradients are averaged automatically; checkpoints/logs come from GPU 0 only). On a 2-GPU machine (e.g. 2×H200):

torchrun --standalone --nproc_per_node=2 -m altasr.training.train \
    --preset medium --audio-root /data/my_corpus \
    --train /data/my_corpus/train.json --val /data/my_corpus/dev.json \
    --out-dir runs/my_model_2gpu

Works on any CUDA GPU generation PyTorch supports (Ampere, Hopper H100/H200, Blackwell, ...): bf16 and TF32 are enabled by capability detection, not hard-coded lists. --batch-size and --num-workers are per GPU. See docs/USAGE.md for scaling guidance.

Multi-GPU inference works too — shard bulk transcription or evaluation across GPUs:

from altasr import TranscriberPool
pool = TranscriberPool.from_pretrained("runs/my_model/best")  # all GPUs
texts = pool.transcribe_batch(list_of_files)   # ~N x faster on N GPUs

altasr-evaluate --checkpoint runs/my_model/best --device all-gpus \
    --audio-root /data/test --metadata /data/test/test.json

TranscriberPool is thread-safe (per-device scheduling), so it drops straight into the web-server examples below in place of Transcriber.

For large corpora, precompute mel features once:

altasr-prepare-data --audio-root /data/my_corpus \
    --metadata /data/my_corpus/train.json --out-dir /data/mels
altasr-train ... --features mel --features-dir /data/mels

Scenario: fine-tune an existing model on new data

altasr-finetune \
    --checkpoint runs/my_model \
    --audio-root /data/new_domain \
    --train /data/new_domain/train.json \
    --val   /data/new_domain/dev.json \
    --out-dir runs/my_model_medical \
    --freeze-layers 8

• The tokenizer is extended with characters found in the new data (old token ids stay stable; the CTC head is resized automatically). Disable with --no-extend-tokenizer.
• --freeze-layers N freezes the feature extractor and the first N encoder blocks — fast, stable adaptation on small datasets.
• Sensible defaults: --lr 1e-4, --epochs 5, short warmup.

Scenario: evaluate and benchmark

# Rich evaluation of one checkpoint
altasr-evaluate --checkpoint runs/my_model \
    --audio-root /data/test --metadata /data/test/test.json --json eval.json

Reported metrics: WER, CER, WER without punctuation, substitution / deletion / insertion rates, and real-time factor (RTF).

# Professional comparison report (Markdown + optional HTML + JSON)
pip install altasr[whisper]
altasr-benchmark \
    --checkpoint "ALTASR medium=runs/my_model" \
    --checkpoint "ALTASR small=runs/small" \
    --whisper small --whisper large-v3 \
    --audio-root /data/test --metadata /data/test/test.json \
    --out benchmark/report.md --html

The report includes a methodology section, a results table (params, WER, CER, error breakdown, RTF), per-system notes, and the hardest utterances — ready to share with stakeholders.

Scenario: serve ALTASR from a web app (FastAPI / Flask / Django)

Load the model once at startup, guard it with a lock, transcribe in a worker thread. Minimal FastAPI service:

import asyncio, os, tempfile
from fastapi import FastAPI, File, HTTPException, UploadFile
from altasr import Transcriber

app = FastAPI()
asr = Transcriber.from_pretrained("runs/my_model/best")
lock = asyncio.Lock()

@app.post("/transcribe")
async def transcribe(file: UploadFile = File(...)):
    suffix = os.path.splitext(file.filename or "a")[1] or ".wav"
    with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp:
        tmp.write(await file.read()); path = tmp.name
    try:
        async with lock:
            return {"text": await asyncio.to_thread(asr.transcribe, path)}
    except Exception as exc:
        raise HTTPException(422, f"could not transcribe: {exc}")
    finally:
        os.unlink(path)

pip install fastapi uvicorn python-multipart
uvicorn app:app --port 8000
curl -F "file=@recording.mp3" http://localhost:8000/transcribe

Complete examples — including a WebSocket live-captioning endpoint, Flask, Django, and production sizing notes — are in docs/USAGE.md.

Checkpoint format

A checkpoint is a plain folder — easy to version, copy, and deploy:

runs/my_model/
├── config.json      # full architecture + training config
├── tokenizer.json   # vocabulary (char or BPE), normalization settings
└── model.pt         # weights

Transcriber.from_pretrained(folder) is all an application needs.

Requirements

• Python ≥ 3.9
• PyTorch ≥ 2.3 (installed automatically; for GPU wheels see pytorch.org)
• FFmpeg is not required — audio decoding uses soundfile / PyAV.

Troubleshooting

Symptom	Fix
ImportError: ... requires PyTorch	pip install torch torchaudio (or reinstall altasr)
BPETokenizer requires sentencepiece	pip install altasr[bpe]
--mic fails to start	pip install altasr[mic]; check OS microphone permissions
Whisper baseline skipped in benchmark	pip install altasr[whisper]
CUDA out of memory during training	--grad-checkpoint (large models); lower --max-batch-seconds; then lower --batch-size + raise --grad-accum
Slow CPU inference	quantize="int8", or export to ONNX
CLI clears my terminal / logs	set ALTASR_NO_CLEAR=1 (clearing is skipped automatically when output is piped)
A term like "DGX" comes out wrong	if it's in your training data: use beam decoding (the default) — the learned LM/lexicon handles it; if never seen: --hotwords "DGX"; for many terms, fine-tune
Only one of my GPUs is used	training: launch with torchrun --standalone --nproc_per_node=<N> -m altasr.training.train ...; inference: --device all-gpus or TranscriberPool
Could not load libtorchcodec / libavutil.so errors	pip install av soundfile (ALTASR's own decoders); no system FFmpeg needed — since v0.2.2 such files are skipped, not fatal
[data] SKIPPING unreadable item warnings	normal for a few bad files in web corpora; 10+ consecutive = wrong --audio-root/--features-dir

License

Apache-2.0 — © YaliLabs / ALTA Project. Source, issues and documentation: https://github.com/yalilabs/altasr