AudioCore

Production-ready audio/video transcription with automatic backend selection

Overview

AudioCore is a Python library for audio and video transcription with automatic backend selection. It seamlessly switches between OpenAI Whisper API (cloud) and faster-whisper (local) based on availability and user preferences, with built-in VAD segmentation, progress tracking, and comprehensive error handling.

Key Features

Supported Backends

Supported Models

Supported Media Formats

AudioCore uses ffmpeg for media processing, supporting:

Installation

Prerequisites

1. Python 3.13+
2. ffmpeg (for media processing)

   # macOS
   brew install ffmpeg
   
   # Ubuntu/Debian
   sudo apt install ffmpeg
   
   # Windows
   winget install ffmpeg
   

From PyPI (Recommended)

pip install audiocore

From Source

git clone https://github.com/seifreed/AudioCore.git
cd AudioCore
python3 -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -e .

Optional Dependencies

# Live microphone capture (real-time transcription)
pip install "audiocore[realtime]"

# Speaker diarization (pyannote.audio)
pip install "audiocore[diarization]"

# For development
pip install "audiocore[dev]"

Quick Start

Python Library

from audiocore import transcribe
from audiocore.types import BackendType, OutputFormat

# Simple transcription (auto-select backend)
result = transcribe("audio.mp3")
print(result.formatted_output)

# With specific options
result = transcribe(
    "video.mp4",
    backend=BackendType.OPENAI,
    language="es",
    output_format=OutputFormat.SRT
)

# Access segments
for segment in result.segments:
    print(f"[{segment.start_time:.2f}s - {segment.end_time:.2f}s] {segment.text}")

# Metadata
print(f"Processing time: {result.processing_time_seconds:.2f}s")
print(f"Backend used: {result.backend_used}")

Command Line Interface

# Basic transcription
audiocore transcribe audio.mp3

# Spanish with SRT output
audiocore transcribe video.mp4 --language es --format srt --output transcript.srt

# Use local faster-whisper
audiocore transcribe podcast.mp3 --backend faster-whisper --model small

# Batch processing
audiocore transcribe *.mp3 --parallel --max-workers 4 --output-dir transcripts/

# Check backend availability
audiocore backends check

CLI Reference

Transcribe Command

audiocore transcribe [OPTIONS] FILE [FILE ...]

Backend Commands

# Check which backends are available
audiocore backends check

# List backend status
audiocore backends list

Config Commands

# Show current configuration
audiocore config show

# Show config file location
audiocore config path

Python API Reference

Main Functions

transcribe()

from audiocore import transcribe
from audiocore.types import BackendType, OutputFormat, ModelSize
from audiocore.models import TranscriptionOptions

# Simple usage
result = transcribe("audio.mp3")

# With TranscriptionOptions
options = TranscriptionOptions(
    backend=BackendType.FASTER_WHISPER,
    model_size=ModelSize.SMALL,
    language="es",
    output_format=OutputFormat.JSON,
    strict_vad=True,  # Fail on VAD errors instead of fallback
)
result = transcribe("audio.mp3", options=options)

# Return type: TranscriptionResult
# - result.segments: List[Segment]
# - result.formatted_output: str (formatted based on output_format)
# - result.processing_time_seconds: float
# - result.backend_used: BackendType
# - result.media_info: MediaInfo

async_transcribe()

import asyncio
from audiocore import async_transcribe

async def main():
    # Single file
    result = await async_transcribe("audio.mp3")
    
    # Multiple files
    results = await async_transcribe(
        ["file1.mp3", "file2.mp4"],
        max_workers=4
    )
    
    for file_result in results:
        if file_result.success:
            print(f"{file_result.path}: OK")
        else:
            print(f"{file_result.path}: Error - {file_result.error}")

asyncio.run(main())

Pipeline Class

from audiocore import Pipeline
from audiocore.pipeline.progress import PipelineStage

def on_progress(stage: PipelineStage, progress: float, message: str):
    print(f"[{stage.value}] {progress*100:.1f}% - {message}")

pipeline = Pipeline(progress_callback=on_progress)
result = pipeline.transcribe("audio.mp3")

Cancellation

from audiocore import Pipeline, CancellationToken
import threading
import time

token = CancellationToken()

def cancel_after_timeout():
    time.sleep(30)  # Cancel after 30 seconds
    token.cancel()

threading.Thread(target=cancel_after_timeout, daemon=True).start()

try:
    result = pipeline.transcribe("video.mp4", cancellation_token=token)
except CancelledError:
    print("Transcription was cancelled")

Configuration

Environment Variables

# OpenAI API
export AUDIOCORE_OPENAI_API_KEY="sk-..."
export AUDIOCORE_OPENAI_TIMEOUT="300"
export AUDIOCORE_OPENAI_MAX_RETRIES="2"

# Faster-Whisper
export AUDIOCORE_FASTER_WHISPER_MODEL="small"
export AUDIOCORE_FASTER_WHISPER_DEVICE="cuda"  # cuda, cpu, or auto
export AUDIOCORE_FASTER_WHISPER_COMPUTE_TYPE="float16"

# VAD settings
export AUDIOCORE_VAD_MIN_SEGMENT_DURATION="0.5"
export AUDIOCORE_VAD_MAX_SEGMENT_DURATION="30.0"
export AUDIOCORE_VAD_SPEECH_THRESHOLD="0.5"

# General settings
export AUDIOCORE_BACKEND="auto"
export AUDIOCORE_LANGUAGE="es"
export AUDIOCORE_MODEL="base"
export AUDIOCORE_OUTPUT_FORMAT="text"
export AUDIOCORE_STRICT_VAD="false"

Configuration File

Create ~/.config/audiocore/config.toml or ./audiocore.toml:

[audiocore]
backend = "auto"
language = "en"
model = "base"
output_format = "text"
strict_vad = false
backend_preference = "auto"  # auto, prefer_local, prefer_cloud

[openai]
api_key = "sk-..."
timeout = 300
max_retries = 2

[faster_whisper]
model = "small"
device = "cuda"
compute_type = "float16"
beam_size = 5
temperature = 0.0

[vad]
min_segment_duration = 0.5
max_segment_duration = 30.0
speech_threshold = 0.5
min_silence_duration_ms = 500

Priority Order

1. CLI arguments (highest priority)
2. Environment variables (AUDIOCORE_*)
3. Configuration file (audiocore.toml)
4. Default values (lowest priority)

Backend Details

OpenAI Whisper API

Requirements:

• OpenAI API key (OPENAI_API_KEY environment variable)
• Internet connection
• Credits in your OpenAI account

Configuration:

[openai]
api_key = "sk-..."
base_url = "https://api.openai.com/v1"  # Optional: for proxies
timeout = 300
max_retries = 2
max_upload_size_mb = 25
chunk_target_size_mb = 24
chunk_min_duration_seconds = 30
chunk_prompt_chars = 1000

AudioCore automatically detects OpenAI uploads larger than max_upload_size_mb, splits them with ffmpeg into chunks below chunk_target_size_mb, transcribes each chunk, and recombines timestamps in the final result.

Advantages:

• High quality transcription
• No local GPU needed
• Fast API response

Limitations:

• Requires API key
• Costs per minute of audio
• Requires internet connection
• Large files require ffmpeg/ffprobe for automatic chunking

Faster-Whisper (Local)

Requirements:

• faster-whisper package (installed by default)
• Optional: CUDA GPU for faster inference
• Model downloaded on first use (cached locally)

Configuration:

[faster_whisper]
model = "small"           # tiny, base, small, medium, large-v3
device = "auto"            # auto, cuda, cpu
compute_type = "float16"   # float16, float32, int8
beam_size = 5
temperature = 0.0

Model Sizes:

Advantages:

• Free (no API costs)
• Works offline
• Privacy (data doesn't leave your machine)
• GPU acceleration support

Note on Device Selection:

• cuda: Use NVIDIA GPU
• cpu: Use CPU (slower but works everywhere)
• auto: Automatically detect best available (CUDA > CPU)
• MPS (Apple Silicon): Currently not supported by CTranslate2, falls back to CPU

VAD (Voice Activity Detection)

AudioCore uses Silero VAD to segment audio into speech chunks before transcription:

from audiocore.vad import VADConfig

config = VADConfig(
    min_segment_duration=0.5,      # Minimum segment length (seconds)
    max_segment_duration=30.0,      # Maximum segment length (seconds)
    speech_threshold=0.5,           # Speech probability threshold (0-1)
    min_silence_duration_ms=500,    # Minimum silence to split (ms)
)

result = transcribe("audio.mp3", vad_config=config)

VAD Behavior:

• If VAD fails and strict_vad=False (default): Falls back to whole-file transcription
• If VAD fails and strict_vad=True: Raises VADError

Error Handling

from audiocore import transcribe
from audiocore.errors import (
    AudioCoreError,
    MediaError,
    BackendUnavailableError,
    TranscriptionError,
    InvalidInputError,
    VADError,
    RateLimitError,
    AuthenticationError,
)

try:
    result = transcribe("audio.mp3")
except InvalidInputError as e:
    print(f"Invalid input: {e.message}")
    print(f"Suggestions: {e.suggestions}")
except MediaError as e:
    print(f"Media processing error: {e.message}")
except BackendUnavailableError as e:
    print(f"No backend available: {e.message}")
    print(f"Available backends: {e.context.get('available_backends')}")
except RateLimitError as e:
    print(f"Rate limited. Retry after: {e.context.get('retry_after')}s")
except AuthenticationError as e:
    print(f"Authentication failed: {e.message}")
except VADError as e:
    print(f"VAD processing failed: {e.message}")
except TranscriptionError as e:
    print(f"Transcription failed: {e.message}")
except AudioCoreError as e:
    print(f"AudioCore error: {e.message}")

Error Types:

Examples

Export to Subtitles

from audiocore import transcribe
from audiocore.types import OutputFormat

# SRT for video players
result = transcribe("video.mp4", output_format=OutputFormat.SRT)
with open("video.srt", "w") as f:
    f.write(result.formatted_output)

# VTT for web players
result = transcribe("video.mp4", output_format=OutputFormat.VTT)
with open("video.vtt", "w") as f:
    f.write(result.formatted_output)

# JSON with metadata
result = transcribe("video.mp4", output_format=OutputFormat.JSON)
with open("video.json", "w") as f:
    f.write(result.formatted_output)

Process Multiple Files

import asyncio
from pathlib import Path
from audiocore import async_transcribe
from audiocore.types import BackendType

async def process_directory(directory: Path):
    audio_files = list(directory.glob("*.mp3")) + list(directory.glob("*.mp4"))
    
    results = await async_transcribe(
        audio_files,
        backend=BackendType.FASTER_WHISPER,
        max_workers=4
    )
    
    for file_result in results:
        if file_result.success:
            output_path = file_result.path.with_suffix(".txt")
            output_path.write_text(file_result.result.formatted_output)
            print(f"✓ {file_result.path.name}")
        else:
            print(f"✗ {file_result.path.name}: {file_result.error}")

asyncio.run(process_directory(Path("videos/")))

Progress Tracking

from audiocore import Pipeline
from audiocore.pipeline.progress import PipelineStage

def on_progress(stage: PipelineStage, progress: float, message: str):
    stages = {
        PipelineStage.PROBING: "📊",
        PipelineStage.EXTRACTING: "🎵",
        PipelineStage.VAD: "🗣️",
        PipelineStage.TRANSCRIBING: "✍️",
        PipelineStage.FORMATTING: "📝",
        PipelineStage.COMPLETE: "✅",
    }
    emoji = stages.get(stage, "⏳")
    print(f"\r{emoji} [{stage.value}] {progress*100:.0f}% - {message}", end="", flush=True)

pipeline = Pipeline(progress_callback=on_progress)
result = pipeline.transcribe("video.mp4")
print()  # New line after progress

Custom VAD Configuration

from audiocore import transcribe
from audiocore.vad import VADConfig
from audiocore.types import BackendType

# More aggressive VAD (faster processing, may miss speech)
fast_vad = VADConfig(
    speech_threshold=0.7,
    min_segment_duration=0.3,
    max_segment_duration=60.0,
)

# More sensitive VAD (slower processing, catches more speech)
sensitive_vad = VADConfig(
    speech_threshold=0.3,
    min_segment_duration=0.5,
    min_silence_duration_ms=200,
)

result = transcribe("audio.mp3", vad_config=fast_vad)

Architecture

audiocore/
├── api/                    # Public API
│   ├── transcribe.py       # Main transcribe functions
│   └── __init__.py         # Public exports
├── backends/               # Backend implementations
│   ├── base.py             # Abstract backend interface
│   ├── openai_backend.py   # OpenAI Whisper API
│   ├── faster_whisper_backend.py  # Local faster-whisper
│   ├── registry.py         # Backend registration
│   ├── availability.py     # Backend availability checking
│   └── selector.py         # Automatic backend selection
├── cli/                    # Command-line interface
│   ├── main.py             # Typer app
│   ├── transcribe.py       # Transcribe command
│   └── config_cmd.py       # Config commands
├── config/                 # Configuration management
│   ├── settings.py         # AppConfig (pydantic-settings)
│   ├── openai_config.py    # OpenAI-specific config
│   ├── faster_whisper_config.py  # Faster-whisper config
│   └── merger.py           # Config priority handling
├── errors/                 # Exception hierarchy
│   ├── base.py             # AudioCoreError base
│   ├── input.py            # InvalidInputError, MediaFormatError
│   ├── media.py            # MediaError
│   ├── backend.py          # BackendUnavailableError
│   ├── transcription.py    # TranscriptionError
│   └── vad.py              # VADError
├── media/                  # Media processing
│   ├── probe.py            # FFprobe wrapper
│   └── extractor.py        # Audio extraction (ffmpeg)
├── models/                 # Data models
│   ├── segment.py          # Segment model
│   ├── media.py            # MediaInfo model
│   └── transcription.py    # TranscriptionResult, TranscriptionOptions
├── output/                 # Output formatters
│   ├── text.py             # Plain text
│   ├── json.py             # JSON format
│   ├── srt.py              # SRT subtitles
│   └── vtt.py              # VTT subtitles
├── parallel/               # Concurrent processing
│   └── files.py            # transcribe_files_concurrent()
├── pipeline/               # Orchestration
│   ├── orchestrator.py     # Main pipeline
│   ├── progress.py         # Progress tracking
│   ├── errors.py           # Pipeline-specific errors
│   └── cancellation.py      # CancellationToken
├── types/                  # Enums and constants
│   └── enums.py            # BackendType, OutputFormat, etc.
├── vad/                    # Voice Activity Detection
│   ├── silero.py           # Silero VAD implementation
│   ├── segments.py         # Segment merging/padding
│   └── config.py           # VADConfig
└── __init__.py             # Package exports

Requirements

Development

Setup

git clone https://github.com/seifreed/AudioCore.git
cd AudioCore
python3 -m venv venv
source venv/bin/activate
pip install -e ".[dev]"

Run Tests

# All tests
pytest

# With coverage
pytest --cov=src/audiocore --cov-report=html

# Specific test file
pytest tests/unit/pipeline/test_orchestrator.py -v

# Integration tests
pytest tests/integration/ -v

Code Quality

# Type checking
mypy src/audiocore

# Linting
ruff check src/audiocore

# Formatting
ruff format src/audiocore

Troubleshooting

"ffmpeg not found"

# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt install ffmpeg

# Windows
winget install ffmpeg

"CUDA not available"

The code automatically falls back to CPU. For CUDA support:

pip install torch --index-url https://download.pytorch.org/whl/cu118

"MPS (Apple Silicon) not working"

faster-whisper uses CTranslate2 which doesn't support MPS. The code automatically falls back to CPU.

"Model download is slow"

Models are cached after first download. To pre-download:

from audiocore.backends.faster_whisper_backend import FasterWhisperBackend
backend = FasterWhisperBackend()
backend._load_model()  # Downloads and caches model

"VAD processing failed"

With strict_vad=False (default), the system falls back to whole-file transcription. Set strict_vad=True to raise errors instead.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Please ensure all tests pass and code coverage remains above 95%.

Software Bill of Materials (SBOM)

AudioCore ships a CycloneDX SBOM of its runtime dependency closure at sbom/audiocore.cdx.json, built by a reproducible generate_sbom.py pipeline and graded by two independent scorers:

• sbomqs: 10.0/10 = 100%, Grade A on the NTIA Minimum Elements standard (2021 and 2025).
• sbom-tools: Grade A (92.6/100 standard profile), with Completeness, Identifiers, Integrity, and Licenses at 100/100.

The SBOM is NTIA-minimum-elements compliant, and CI (.github/workflows/sbom.yml) fails if either rating regresses (sbom-tools below Grade A, or sbomqs NTIA below 100/A). See sbom/README.md for the full breakdown and make sbom to regenerate.

Roadmap

Completed (v1.0)

• Foundation (error hierarchy, types, models)
• Configuration system (env, TOML, CLI)
• Media ingestion (probe, extract)
• VAD processing (Silero)
• Backend abstraction
• OpenAI Whisper API backend
• Faster-Whisper local backend
• Automatic backend selection
• Pipeline orchestration
• CLI and public API
• Multiple output formats (text, JSON, SRT, VTT)
• Progress tracking
• Cancellation support
• Concurrent batch processing
• Comprehensive error handling

Future (v2.0)

• Real-time transcription
• Speaker diarization
• WebAssembly support
• Streaming API
• Custom VAD models
• Word-level timestamps
• Translation API

Word-level timestamps

Pass word_timestamps=True (or --word-timestamps on the CLI) to attach per-word timing to every segment. Works with both backends — faster-whisper emits native word timestamps with confidence; the OpenAI backend requests timestamp_granularities. Words are included in JSON output.

from audiocore import transcribe
from audiocore.models import TranscriptionOptions
from audiocore.types import OutputFormat

result = transcribe(
    "audio.mp3",
    options=TranscriptionOptions(word_timestamps=True, output_format=OutputFormat.JSON),
)
for segment in result.segments:
    for word in segment.words or []:
        print(f"[{word.start_time:.2f}-{word.end_time:.2f}] {word.word}")

audiocore transcribe audio.mp3 --word-timestamps --format json

Translation API

Pass task=TranscriptionTask.TRANSLATE (or --translate on the CLI) to translate speech into English. faster-whisper uses its translate task; the OpenAI backend routes to the dedicated translations endpoint. (Whisper's translate task always targets English regardless of the source language.)

from audiocore import transcribe
from audiocore.models import TranscriptionOptions
from audiocore.types import TranscriptionTask

result = transcribe(
    "entrevista_es.mp3",
    options=TranscriptionOptions(task=TranscriptionTask.TRANSLATE),
)
print(result.formatted_output)  # English translation

audiocore transcribe entrevista_es.mp3 --translate

Custom VAD models

Two extension points replace the bundled Silero VAD:

1. A custom model file — point VADConfig.model_path at a Silero-format TorchScript model:

   from audiocore.vad import VADConfig
   from audiocore.config import AppConfig
   
   config = AppConfig(vad=VADConfig(model_path="my_vad.jit"))
   

2. A fully custom detector — implement the VADModel protocol (detect_file(audio_path, config) -> list[(start, end, confidence)]) and pass it to the pipeline:

   from audiocore import transcribe
   from audiocore.vad import VADModel
   
   class EnergyVAD:
       def detect_file(self, audio_path, config=None):
           return [(0.0, 4.2, 1.0)]  # your detection logic
   
   result = transcribe("audio.mp3", vad_model=EnergyVAD())
   

Streaming API

stream_transcribe() yields Segment objects as they are produced instead of returning one TranscriptionResult at the end. With the local faster-whisper backend, segments arrive incrementally as the model decodes; other backends yield once decoding completes. Pipeline.stream_transcribe() is the class-level equivalent.

from audiocore import stream_transcribe

for segment in stream_transcribe("podcast.mp3", backend="faster_whisper"):
    print(f"[{segment.start_time:.2f}s] {segment.text}")

Real-time transcription

Transcribe a live audio stream utterance by utterance. An energy gate groups incoming chunks into utterances and hands each to the backend as the speaker pauses. Microphone capture needs the optional realtime extra (pip install audiocore[realtime]); any iterable of float32 chunks works as a custom AudioSource without it.

from audiocore.realtime import MicrophoneSource, RealtimeTranscriber, transcribe_realtime

# Convenience: auto-selects a backend from config.
for segment in transcribe_realtime(MicrophoneSource(), backend=None):
    print(segment.text)

# Or drive it explicitly with any backend.
from audiocore.backends import FasterWhisperBackend
transcriber = RealtimeTranscriber(FasterWhisperBackend())
for segment in transcriber.stream(MicrophoneSource()):
    print(f"[{segment.start_time:.1f}s] {segment.text}")

Speaker diarization

Pass a Diarizer to label each segment with a speaker (Segment.speaker, included in JSON output). The optional pyannote backend needs the diarization extra (pip install audiocore[diarization]) and a Hugging Face token for the gated model; any object implementing diarize(audio_path) -> list[SpeakerTurn] works without it.

from audiocore import transcribe
from audiocore.diarization import PyannoteDiarizer

result = transcribe("meeting.wav", diarizer=PyannoteDiarizer(auth_token="hf_..."))
for segment in result.segments:
    print(f"{segment.speaker}: {segment.text}")

WebAssembly support

The transcription engine depends on native components (ffmpeg, PyTorch, CTranslate2) that cannot run in a browser, but AudioCore's models and output formatters are pure Python. audiocore.wasm is the Pyodide-safe entry point — importing it pulls in no torch, ctranslate2, ffmpeg, openai, or backend code — so a browser app can render text/JSON/SRT/VTT from timing+text produced elsewhere (e.g. a JS-side whisper.cpp WASM build or an API response).

from audiocore.wasm import format_transcript, is_wasm

segments = [
    {"start_time": 0.0, "end_time": 1.5, "text": "Hello"},
    {"start_time": 1.5, "end_time": 3.0, "text": "world"},
]
srt = format_transcript(segments, "srt")   # also: "text", "json", "vtt"
print(is_wasm())                            # True under Pyodide/Emscripten

Support the Project

If you find AudioCore useful, consider supporting its development:

License

This project is licensed under the MIT License - see the LICENSE file for details.

Attribution Required:

• Author: Marc Rivero | @seifreed
• Repository: github.com/seifreed/audiocore

_{Made with dedication for the audio/video transcription community}