ALTASR

Speech recognition for Kinyarwanda — with English, French and Swahili code-switching — by Yali Labs (ALTA Project).

ALTASR transcribes Kinyarwanda speech as it is actually spoken in Rwanda: courtrooms, clinics, government offices, markets, radio — including the English and French words and phrases speakers mix in mid-sentence. It runs offline on your own hardware, from a Raspberry-class CPU to a multi-node GPU cluster, and ships both an offline (highest accuracy) and a streaming (live captioning) model family behind one API.

pip install altasr

Extras: altasr[mic] (microphone streaming), altasr[onnx] (ONNX export/runtime), altasr[bpe] (subword tokenizer support).

Quickstart

from altasr import ASR

asr = ASR.from_pretrained("path/to/checkpoint")     # a checkpoint folder
print(asr.transcribe("recording.wav").text)

Something wrong? Run the doctor first

altasr doctor checks your Python/PyTorch/CUDA/audio stack and tells you exactly what is broken and how to fix it; --fix applies the safe fixes (the right PyTorch wheel for your GPU, missing audio backends) after showing you the plan:

altasr doctor          # diagnose: environment, GPU, audio backends
altasr doctor --fix    # repair a broken PyTorch/CUDA install

It is the first thing to reach for when anything misbehaves — wrong device, cannot decode mp3, NCCL errors, slow inference.

Transcribing

One call handles every input kind:

from altasr import ASR
asr = ASR.from_pretrained("path/to/checkpoint")

out = asr.transcribe("recording.wav")               # a file (any format)
print(out.text)

outs = asr.transcribe(["a.wav", "b.wav"])           # a batch of files

import numpy as np                                   # a numpy array
pcm = np.zeros(16000, dtype=np.float32)              # 1 s of audio @16 kHz
out = asr.transcribe(pcm, sample_rate=16000)

with open("recording.wav", "rb") as fh:              # raw bytes / file obj
    out = asr.transcribe(fh.read())

out = asr.transcribe("https://example.com/audio.mp3")   # a URL

Whole directories go through the CLI:

altasr transcribe ./recordings --recursive --output transcripts/

Sample rates, channel counts and container formats are handled for you (mp3/m4a/ogg/flac/wav; resampling is automatic).

Long audio (hours)

Give transcribe() a one-hour hearing; internally it voice-activity segments the audio, decodes overlapping chunks, and merges them with a local-agreement algorithm so words at chunk boundaries are not duplicated or lost. Memory stays flat (bounded by the chunk size, not the file length) — about 2 GB RSS for CPU decoding of arbitrarily long files.

out = asr.transcribe("recording.wav", speakers="off")
print(out.duration, "seconds transcribed")

Live captioning (streaming)

Streaming checkpoints caption as the audio arrives — partials update live, finals are stable:

from altasr import ASR
asr = ASR.from_pretrained("path/to/streaming-checkpoint")

with asr.stream() as session:
    for chunk in audio_chunks:            # bytes or numpy, as they arrive
        for hyp in session.push(chunk, sample_rate=16000):
            print("final" if hyp.is_final else "partial", hyp.text)
    print("done:", session.finish())

partial lines may still change; final lines never do — render partials in grey and replace them, append finals. Session state is serializable, so a dropped connection can resume where it stopped. Offline checkpoints refuse stream() with a clear message (they still handle long files — see above); microphone capture is altasr stream --mic (with pip install altasr[mic]).

Speaker-attributed dialogue transcription

ALTASR does not just transcribe what was said — it can tell you who said it. Speaker detection is automatic; labels only appear when more than one speaker is actually present:

out = asr.transcribe("recording.wav")                # auto-detect
print(out.text)                                      # "SPEAKER 1: ..." style
out = asr.transcribe("recording.wav", speakers=3)    # force a known count
print(out.as_dialogue("srt"))                        # txt | json | srt | vtt | court

Court example — a hearing with a judge, a prosecutor and a witness: asr.transcribe("hearing.wav", speakers=3).as_dialogue("court") produces a numbered, time-stamped record with a speaker column, ready for review.

Clinic example — a consultation: asr.transcribe("consultation.wav", speakers=2) separates clinician and patient turns so the note-taker only corrects, never untangles.

Benchmarking on your own recordings — the full guide

Two tools, two purposes:

• ConversationBenchmark (Python/Jupyter) — review and score real multi-turn, code-switched conversations: speaker turns with timestamps next to an audio player, per-file and corpus WER/CER, real-time factor.
• altasr-benchmark (CLI) — a publishable Markdown/HTML report comparing several checkpoints (and optionally Whisper) on a labelled test set.

Step 1 — collect the recordings in one folder (visits/a.wav, visits/b.mp3, ...). Any length: long files are chunked automatically.

Step 2 (optional but recommended) — write references so you get WER/CER, in ANY of these forms: a dict in code, a JSON file ({"a": "reference text...", "b": "..."} — keys are file names or stems), or a folder of a.txt, b.txt. Write them the natural way — scoring passes both sides through the model's own normalizer with punctuation stripped, so casing/punctuation/number style never count as errors.

Step 3 — run it (in Jupyter, bench.show adds an audio player):

from altasr.benchmark import ConversationBenchmark

bench = ConversationBenchmark("path/to/checkpoint", device="cpu",
                              hotwords=["paracetamol", "umuganga"])
results = bench.run(["a.wav", "b.wav"],
                    refs={"a": "aba baba", "b": "ba ab"})
bench.show(results[0], audio=False)   # turns + timestamps (+ player)
bench.table(results)                  # pandas summary per file
bench.save(results, "benchmark_results.json")

run() prints per-file progress, then one corpus-level WER x% | CER y% over every referenced file plus the mean real-time factor. Each result dict carries the speaker turns, the formatted transcript, and (with a reference) the raw scored transcript so you can inspect every disagreement.

Step 4 — compare checkpoints. Repeat with each candidate (e.g. best/ vs the altasr-avg best-5 average) and keep the JSONs; or generate one professional report over a labelled set in a single command:

altasr-benchmark \
    --checkpoint "ALTASR v1"=path/to/best \
    --checkpoint "ALTASR v1 avg"=path/to/avg \
    --audio-root /data --metadata /data/test.json \
    --out benchmark/report.md --html

The report includes methodology, WER/CER/wer_nopunct, substitution/deletion/insertion rates, RTF, and the hardest utterances per system as REF/HYP pairs.

Name the voices once and ALTASR labels them by name in every later recording:

altasr enroll add --name "Judge Mukamana" --audio judge_sample.wav
altasr enroll list
altasr enroll delete --name "Judge Mukamana"

High-stakes use — read this. ALTASR output is a draft for human review. It is not a certified record. Speech recognition makes mistakes — names, numbers, negations — and speaker attribution can be wrong, especially with overlapping speech. In legal, medical, and other high-stakes settings, a qualified human must review and certify every transcript before it is relied on. The court output format prints this disclaimer on every document it renders.

Privacy — voice profiles are biometric data. altasr enroll stores voice embeddings (not audio) encrypted at rest with a passphrase you control (ALTASR_PROFILE_PASSPHRASE). They identify a person and are subject to biometric-data law in many jurisdictions: collect consent, set a retention period, and delete profiles with altasr enroll delete when they are no longer needed. To run with no persistence at all, simply never enroll anyone — automatic diarization ("SPEAKER 1/2/…") keeps nothing between calls.

Multi-GPU and multi-node inference

Point a batch of files at every GPU in the machine — or several machines:

outs = asr.transcribe(files, devices="all")          # every local GPU
outs = asr.transcribe(files, devices="cuda:0,2")     # a subset

altasr transcribe ./folder --recursive --gpus all --output out/

Multi-node needs no scheduler: run the same command on each node with a shared --work-dir — a file-based work queue hands out items exactly once, survives worker crashes (stale claims are reclaimed), and resumes where it stopped if you re-run the same command after an interruption. Throughput scales close to linearly with GPUs for file batches, because items are independent.

Output format

Every call returns a structured TranscriptionOutput:

out = asr.transcribe("recording.wav", speakers="off")
out.text                # the transcript
out.segments            # [Segment(start, end, text, speaker, confidence)]
out.words               # word-level timestamps + confidences
out.speakers            # SpeakerInfo(count, labels, mode, confidence)
out.language_spans      # [LanguageSpan(lang, start_word, end_word)] rw/en/fr/sw
out.to_dict()           # JSON-ready
out.save("transcript.json")

Options

option	what it does
speakers="auto"|N|"off"	speaker attribution: detect, force a count, or plain text
hotwords=[...]	bias decoding toward domain terms (see below)
decoder="auto"|"greedy"|"beam", beam_size=	speed/accuracy trade-off
lm_weight=	weight of the built-in corpus language model in beam search
sample_rate=	required for raw numpy/bytes input
devices=, work_dir=	multi-GPU / resumable batch jobs
formatting=False	return raw lowercase output instead of the formatted text
VAD/chunking	long-file segmentation is automatic; altasr-vad exposes the segmenter standalone

Readable output, automatically

Checkpoints ship with learned output formatting, applied without any extra work on your side: proper names are truecased ("kigali" → "Kigali", "diane" → "Diane"), sentences are capitalized and punctuated — pauses in the speech decide where sentences end — and numbers, RWF amounts, phone numbers, and dates are written the way people write them:

out = asr.transcribe("recording.wav", speakers="off")
print(out.text)        # "Muraho Diane, turi i Kigali."  — not
                       # "muraho diane turi i kigali"

Streaming works the same way: provisional partials stay raw (they may still change), finalized lines arrive formatted. Pass formatting=False anywhere to get the raw output — benchmarking tools do this automatically so accuracy numbers stay comparable. The ITN rules are also usable standalone:

from altasr.text.itn import inverse_normalize
print(inverse_normalize("yishyuye ibihumbi bitanu amafaranga"))
# -> "yishyuye RWF 5,000"

Complete parameter reference

Every parameter of the inference API. (transcribe_batch accepts the same decoding/formatting parameters as transcribe.)

ASR.from_pretrained(checkpoint, ...)

parameter	default	meaning
checkpoint	required	checkpoint folder path
device	"auto"	auto | cpu | cuda | cuda:N
quantize	""	"int8" = dynamic int8 quantization (CPU only): 3–4× smaller, 1.5–3× faster
adapter	None	path to a fine-tuned adapter file to load on top of the base model

asr.transcribe(audio, ...)

parameter	default	meaning
audio	required	file path, URL, numpy array, torch tensor, bytes, file object, or a list of paths
sample_rate	None	required for raw numpy/tensor/bytes input
speakers	"auto"	speaker attribution: "auto" (labels only when >1 detected) | int (force a count) | "off" (plain text) | a profiles store
decoder	"auto"	auto (best available) | greedy (fastest) | beam
beam_size	0	beam width when beam decoding (0 = the default width, 8)
hotwords	None	domain terms to bias toward: a list, comma string, or "@file"
hotword_bonus	3.0	per-token score bonus along hotword matches (raise if terms still lose, lower if they over-fire)
lm_weight	None	weight of the checkpoint's built-in language model in beam search (None = default 0.3; 0 disables)
oov_correction	True	confidence-gated fixing of out-of-vocabulary words against the learned lexicon
formatting	None	None = apply the checkpoint's truecasing/punctuation/ITN automatically; False = raw lowercase output
batch_size	8	decode batch size for lists of inputs
resample	True	auto-resample non-16 kHz input
progress	None	callback receiving completion fraction 0–1
devices	None	spread a LIST of files across GPUs: "all" | "cuda:0,2" | int | "cpu"
work_dir	None	shared work directory: makes batch jobs resumable and enables multi-node

asr.stream(...) session (streaming checkpoints)

parameter	default	meaning
endpoint_blank_frames	20	finalize an utterance after this many consecutive blank frames
endpoint_silence_s	0.8	finalize after this much trailing silence
max_utterance_s	30.0	force-finalize utterances longer than this
session.push(chunk, sample_rate=None)	–	feed audio (bytes/numpy); returns hypotheses with .text and .is_final
session.finish()	–	flush and return the last final text

altasr transcribe (CLI) — --checkpoint DIR (or env ALTASR_CHECKPOINT), --recursive, --speakers auto|N|off, --format txt|json|srt|vtt|tsv|court, --output PATH, --hotwords LIST|@file, --profiles STORE, --device, --gpus all|N|i,j, --work-dir DIR, --nodes N, --node-rank R. Full help: any command + --help.

Fine-tune on your own recordings

Adapt a base checkpoint to your domain (a courtroom, a clinic, a call center) with altasr-finetune — you supply audio + transcripts in any common metadata format (JSON/JSONL/CSV with audio_path + text columns; see altasr-finetune --help for the accepted layouts):

altasr-finetune --checkpoint path/to/checkpoint \
    --audio-root ./my_recordings --train my_transcripts.json \
    --val my_dev.json --out-dir runs/my-domain

flag	default	meaning
--checkpoint DIR	required	the base model to adapt
--train META / --val META	required / –	your transcript metadata file(s), repeatable
--audio-root DIR	–	folder your audio paths are relative to
--freeze-layers N	0	keep the first N encoder layers frozen (small datasets: freeze more)
--no-extend-tokenizer	off	don't add new characters found in your data
--limit N	0	fine-tune on only the first N utterances (quick trials)
--epochs, --lr, --batch-size	sensible defaults	standard knobs; small data wants few epochs and a low learning rate
--set model.lora_rank=8	off	LoRA mode: produces a small adapter.pt instead of a full model — load it with from_pretrained(..., adapter=...)

Every additional flag (there are many, all documented): altasr-finetune --help. Rule of thumb: with under an hour of audio, prefer LoRA adapters or --freeze-layers; with tens of hours, full fine-tuning wins.

Domain adaptation without training

Two mechanisms, no GPUs required:

Hotwords — hand the decoder your domain terms (case names, drug names, place names) per call or from a file:

out = asr.transcribe("recording.wav", speakers="off",
                     hotwords=["diyabete", "insuline"], decoder="beam")

altasr transcribe ./clinic --hotwords @data/lexicons/health.txt

Adapters — a domain adapter is a small file (a few MB) that specializes a base checkpoint; load it at startup:

asr = ASR.from_pretrained("path/to/checkpoint", adapter="health.adapter")

Available checkpoints

Each release publishes a table like the one below alongside the download (per-condition WER is measured on held-out Rwandan speech; RTF = seconds of audio processed per second of compute, higher is faster):

checkpoint	arch	size	WER clean	WER code-switched	RTF (CPU)	RTF (GPU)
altasr-large	offline	~430 MB	see release notes	see release notes	~1x	~30x
altasr-streaming	streaming	~180 MB	see release notes	see release notes	~2x	~50x

A checkpoint is a folder (model.pt, tokenizer.json, config.json, meta.json); pass the folder path to from_pretrained. meta.json declares the architecture and capabilities, so the API auto-detects what each checkpoint can do.

CLI reference

command	purpose
altasr doctor [--fix] [--distributed]	diagnose/repair the environment
altasr transcribe <path> [--recursive] [--gpus all] [--hotwords ...]	transcribe files/folders
altasr stream [--mic | file]	live captioning
altasr enroll add/list/delete	named speaker profiles (encrypted)
altasr-vad segments <file>	voice-activity segmentation standalone
altasr-eval --checkpoint <dir> ...	measure accuracy on labelled audio
altasr-bench	latency/throughput benchmark
altasr-export	package a checkpoint for deployment (ONNX + parity verify, int8, non-Python bundle)
altasr-serve	REST + jobs + WebSocket captioning server with /health

Every command prints full help with --help.

Integration

Python — everything above. CLI — everything above.

REST / WebSocket server (altasr-serve) — the full guide

Built in, standard library only. Start it:

altasr-serve --checkpoint path/to/checkpoint --port 8080
# flags: --host 0.0.0.0 | --port 8080 | --device auto|cpu|cuda:N
#        --diarize (speaker labels on /stream) | --speaker-embedder PATH
# or set ALTASR_CHECKPOINT instead of --checkpoint

The one thing to know about request bodies: POST /transcribe and POST /jobs take the raw audio file bytes as the entire body — there is no JSON wrapper, no form fields, no keys. Any container the package decodes works (wav/mp3/m4a/ogg/flac); sample rate and channels are handled server-side. Decoding options (device, checkpoint) are chosen when you start the server, not per request.

1. GET /health — liveness (load balancers, Docker healthchecks):

→ 200 {"status": "ok", "arch": "offline", "device": "cuda:0",
       "streaming": false, "jobs": {"queued": 0, "running": 0, ...}}

2. POST /transcribe — synchronous; best for clips up to a few minutes:

curl --data-binary @recording.wav http://localhost:8080/transcribe

With Postman: method POST, URL http://localhost:8080/transcribe, Body → binary → Select File → choose the audio file. (Do not use form-data or raw JSON.) The response is JSON:

→ 200 {"text": "Muraho Diane, turi i Kigali.",
       "is_dialogue": false, "duration": 12.4, "speakers": null,
       "segments": [{"start": 0.0, "end": 4.1, "text": "...",
                     "speaker": null, "overlap": false,
                     "confidence": 0.94}, ...],
       "words": [{"word": "muraho", "start": 0.12, "end": 0.55,
                  "speaker": null, "confidence": 0.97}, ...]}
→ 400 {"error": "empty body — send audio bytes"}
→ 422 {"error": "<what went wrong decoding this audio>"}

3. POST /jobs + GET /jobs/<id> — asynchronous, for long recordings (hearings, full consultations). Submit the same binary body; you get an id back immediately, then poll:

curl --data-binary @hearing.mp3 http://localhost:8080/jobs
# → 202 {"job_id": "a1b2c3", "poll": "/jobs/a1b2c3"}
curl http://localhost:8080/jobs/a1b2c3
# → {"status": "queued" | "running"}          while working
# → {"status": "done", "result": {...same JSON as /transcribe...}}
# → {"status": "failed", "error": "..."}

4. GET /stream — WebSocket live captioning (needs a streaming checkpoint; Postman supports WebSocket requests, or use any WS client):

1. connect to ws://localhost:8080/stream;
2. optionally send ONE text message with the audio format: {"sample_rate": 16000, "format": "s16"} ("s16" = 16-bit PCM little-endian, the default; "f32" = float32);
3. send raw PCM chunks as binary messages as the audio arrives;
4. receive JSON events: {"type": "partial"|"final", "text": "...", "speaker": "SPEAKER 1"|null, "speaker_final": true|false} — render partials as provisional (grey), replace them; append finals;
5. send {"eof": true} as a text message to flush the last words and close.

Your own web app (FastAPI/Flask/Django) — load once at startup, transcribe per request; ASR is thread-safe for inference:

from fastapi import FastAPI, UploadFile
from altasr import ASR

app = FastAPI()
asr = ASR.from_pretrained("path/to/checkpoint")     # once, at startup

@app.post("/transcribe")
async def transcribe(file: UploadFile):
    return asr.transcribe(await file.read()).to_dict()

ONNX — altasr-export path/to/checkpoint --format onnx --out model.onnx --verify 100 exports for runtimes without PyTorch and verifies numerical parity against PyTorch (pip install altasr[onnx]). --format bundle writes a fully self-contained folder — ONNX graph, feature/vocab spec, tokenizer, C example — for embedding ALTASR in non-Python systems via the ONNX Runtime C API.

Docker — a Dockerfile and docker-compose.yml (with a health check) ship in the repository:

docker compose up          # serves your ./ckpt folder on :8080

Non-Python example (curl against altasr-serve):

curl --data-binary @recording.wav http://localhost:8080/transcribe
curl http://localhost:8080/health

Performance and hardware

• CPU-only works. Quantize for 3–4× smaller and 1.5–3× faster with near-identical accuracy:

from altasr import ASR
asr = ASR.from_pretrained("path/to/checkpoint", device="cpu",
                          quantize="int8")

• GPU: any CUDA device with ≥4 GB memory decodes the large model; bf16 is used automatically on Ampere and newer.
• Memory: long files are chunked — RSS stays flat regardless of file length. Batch decoding scales with batch_size.
• Real-time streaming needs roughly one modern CPU core per session; a single GPU serves tens of concurrent sessions (the session router enforces per-device caps and reports latency percentiles).

Troubleshooting

symptom	do this
Poor accuracy	Checklist: right checkpoint for the domain? 16 kHz+ source? try decoder="beam"; add hotwords for domain terms; check out.words confidences to find where it fails.
CUDA out of memory	lower batch_size; decode on CPU; for very long files nothing is needed (chunking bounds memory). Run altasr doctor to confirm the GPU is healthy.
Slow inference	altasr doctor (is the GPU actually used?); quantize on CPU; batch files instead of looping; devices="all" for many files.
Cannot read mp3/m4a	altasr doctor --fix installs the audio backend.
Wrong/garbled language	the checkpoint is Kinyarwanda-centric; heavy non-rw speech needs the code-switching checkpoint from the releases page.
stream() raises on my checkpoint	offline checkpoints do not stream; use a *-streaming checkpoint (the error message says exactly this).

FAQ

Which languages? Kinyarwanda first-class, including code-switched English and French insertions; Swahili support is expanding with the corpus.

Does audio leave my machine? Never. ALTASR runs fully offline; there is no telemetry.

Can I use it commercially? Yes — Apache-2.0.

How do I make it learn my domain's vocabulary? Hotword lists and adapters (above) — no training needed.

License and citation

Apache-2.0. © Yali Labs (ALTA Project).

@software{altasr,
  title  = {ALTASR: Kinyarwanda speech recognition with code-switching},
  author = {{Yali Labs, ALTA Project}},
  year   = {2026},
  url    = {https://github.com/yalilabs/altasr}
}

Support: open a GitHub issue, or email the ALTA Project team.