fasr-service-fastapi

FastAPI service plugin for fasr. This package provides the concrete online service implementation while the core fasr package keeps only service base classes and registries.

Features

• Demo web page:
  • GET / (same as GET /demo)
• Optional batch transcription HTTP endpoints:
  • POST /transcribe
  • POST /inference
• Optional realtime websocket endpoints:
  • GET /v1/realtime
  • GET /api-ws/v1/realtime
• Health endpoint:
  • GET /health
• Config-driven model and router initialization through fasr registries

Realtime API Docs

• English: REALTIME_API.md
• Chinese: REALTIME_API_ZH.md

The realtime router supports runtime turn_detection updates through the VAD model's apply_turn_detection(...) hook. At the moment, the streaming VAD models with documented runtime support are:

• fsmn_online
• stream_silero

Client-side realtime VAD updates use the websocket session.update event. The server replies with session.updated when the update is accepted. Example:

{
  "type": "session.update",
  "session": {
    "turn_detection": {
      "type": "server_vad",
      "silence_duration_ms": 300,
      "threshold": 0.5,
      "prefix_padding_ms": 100
    }
  }
}

The currently documented server_vad runtime fields are:

• silence_duration_ms
• threshold
• prefix_padding_ms

See REALTIME_API.md and REALTIME_API_ZH.md for the full event contract and field semantics.

Registered entry points

Configuration

The service owns model instances under [service.models]. Each router owns its own model_map, which maps router argument names to model names from [service.models]. Routers are enabled only when their config block exists, so omitting [service.transcribe_router] skips batch HTTP routes and omitting [service.realtime_router] skips realtime websocket routes. Configuring neither router is invalid and fasr-fastapi serve will fail fast with a clear error.

The config has four layers:

• [service]: service-level knobs such as host, port, and debug_logging.
• [service.models.*]: model instances owned by the service.
• [service.<router>.model_map]: which service-owned models are wired into a router.
• [service.<router>.component_configs] and [service.<router>.pipe_configs]: runtime tuning for transcribe components and their threaded Pipe wrappers.

When using nested dictionary sections such as component_configs or pipe_configs, keep the empty parent section in the file first:

[service.transcribe_router.component_configs]
[service.transcribe_router.component_configs.detector]
num_threads = 4

Without the parent section, confection may fail to parse the nested config.

Minimal transcribe-only config:

[service]
@services = "fastapi.v1"

[service.models.paraformer]
@asr_models = "paraformer"

[service.models.marblenet]
@vad_models = "marblenet"

[service.transcribe_router]
@service_routers = "transcribe.v1"

[service.transcribe_router.model_map]
vad_model = "marblenet"
asr_model = "paraformer"

Full example with punctuation and tunable runtime parameters:

[service]
@services = "fastapi.v1"
host = "0.0.0.0"
port = 8000
debug_logging = false

[service.models.qwen3_asr]
@asr_models = "qwen3asr"
size = "small"

[service.models.marblenet]
@vad_models = "marblenet"

[service.models.ct_transformer]
@punc_models = "ct_transformer"

[service.models.fsmn_online]
@vad_models = "fsmn_online"

[service.transcribe_router]
@service_routers = "transcribe.v1"

[service.transcribe_router.model_map]
vad_model = "marblenet"
asr_model = "qwen3_asr"
punc_model = "ct_transformer"

[service.transcribe_router.component_configs]

[service.transcribe_router.component_configs.loader]
timeout_seconds = 60.0
max_concurrency = 32

[service.transcribe_router.component_configs.detector]
num_threads = 4
max_segment_duration = 30.0

[service.transcribe_router.component_configs.recognizer]
num_threads = 2
batch_size_s = 30

[service.transcribe_router.component_configs.sentencizer]
num_threads = 2

[service.transcribe_router.pipe_configs]

[service.transcribe_router.pipe_configs.detector]
batch_size = 8
batch_timeout = 0.2
verbose = false

[service.transcribe_router.pipe_configs.recognizer]
batch_size = 8
batch_timeout = 0.2
verbose = false

[service.transcribe_router.pipe_configs.sentencizer]
batch_size = 8
batch_timeout = 0.2
verbose = false

[service.realtime_router]
@service_routers = "realtime.v1"

[service.realtime_router.model_map]
vad_model = "fsmn_online"
asr_model = "qwen3_asr"

Set debug_logging = true if you want to keep the service's DEBUG logs during development.

Router entry points are factory functions. For example, transcribe_entrypoint creates default AudioLoader, VoiceDetector, and SpeechRecognizer instances when the config only provides model_map. This avoids requiring users to spell out internal router components in simple service configs. When punc_model is present in the transcribe router model_map, the router also creates a default SpeechSentencizer and adds it after the recognizer.

pipe_configs forwards runtime knobs to the underlying Pipe wrappers. Each transcribe pipe can currently configure:

• batch_size
• batch_timeout
• verbose

component_configs forwards keyword arguments to each component's setup(). This is the place to tune component-native knobs such as:

• loader.max_concurrency
• loader.timeout_seconds
• detector.num_threads
• detector.max_segment_duration
• recognizer.num_threads
• recognizer.batch_size_s
• sentencizer.num_threads

CLI

Install the service plugin through the root project extra:

uv sync --extra service

Or install the plugin package directly:

pip install fasr-service-fastapi

Generate and run a default config:

fasr-fastapi init --cfg run.cfg
fasr-fastapi serve --cfg run.cfg

Inspect or write the default config without starting the server:

fasr-fastapi serve --print_default_config true
fasr-fastapi serve --write_default_config run.cfg

The legacy fasr init and fasr serve commands still forward to this plugin CLI for now, but fasr-fastapi is the supported command surface going forward.

The same default config is available from:

from fasr_service_fastapi import FastAPIASRService

config = FastAPIASRService.model_construct().get_default_config()

Realtime Protocol

The websocket protocol is documented separately in REALTIME_API.md.

Development

Install from the repository root:

uv sync

Run service tests:

uv run pytest tests/test_online_service.py -q

Build the plugin:

uv build --package fasr-service-fastapi