ASR Concurrent Stream Server

Concurrent streaming gRPC server and client for Qwen3-ASR.

Installation

pip install asr-concurrent-stream

For client-side audio loading and VAD segmentation:

pip install asr-concurrent-stream[client]

For development (regenerating protobuf code):

pip install asr-concurrent-stream[dev]

Note: The server requires vllm and transformers to run the Qwen3-ASR model. These are installed automatically with the package.

Quick Start

Start the server

asr-concurrent-server

The server reads configuration from environment variables:

Run the client

asr-concurrent-client --audio path/to/audio.wav --language Italian

Dependencies

Optional client dependencies: soundfile, torch (for Silero VAD segmentation).

Docker image for the Qwen3-ASR concurrent streaming gRPC server.

Image Composition

Installation Sequence

1. System dependencies — curl, git, wget, build-essential, ffmpeg, libsox-*, libsndfile1, libopus0, libffi-dev.
2. Python 3.12 — installed via deadsnakes PPA, venv created at /opt/venv, binaries symlinked to /usr/local/bin.
3. pip — upgraded to latest pip, setuptools, wheel.
4. qwen-asr[vllm] — installs the full Qwen3-ASR stack with vLLM extras (transformers, vllm, torch, accelerate, librosa, soundfile, etc. with pinned versions).
5. flash-attention — prebuilt wheel for CUDA 12.8 + PyTorch 2.9 + Python 3.12.
6. Project files — ASR_Concurrent_Stream/ copied to /app/ASR_Concurrent_Stream/.
7. Model download — Qwen/Qwen3-ASR-1.7B downloaded from HuggingFace to /app/models/Qwen3-ASR-1.7B.
8. Launcher patch — run_asr_grpc_server.sh is updated:
   • PORT → 8002
   • MODEL_PATH → /app/models/Qwen3-ASR-1.7B
   • QWEN3_ASR_PATH → site-packages qwen_asr path

Build & Run

# Build
docker build -t qwen3-asr-server:latest .

# Run (map host port 8002 to container port 8002)
docker run -d \
    --name qwen3-asr-server \
    --gpus all \
    -p 8002:8002 \
    --shm-size=16g \
    --ulimit memlock=-1 \
    --restart unless-stopped \
    qwen3-asr-server:latest

# View logs
docker logs -f qwen3-asr-server

# Stop & remove
docker stop qwen3-asr-server && docker rm qwen3-asr-server

Server Configuration

Hard-coded defaults in server/run_asr_grpc_server.sh:

Project Structure (in image)

/app/
├── ASR_Concurrent_Stream/
│   ├── proto/              # gRPC protobuf definitions
│   ├── server/
│   │   ├── asr_grpc_server.py
│   │   ├── _launch_server.py
│   │   ├── stream_manager.py
│   │   ├── inference_coordinator.py
│   │   └── run_asr_grpc_server.sh
│   └── client/
│       └── example_client.py
└── models/
    └── Qwen3-ASR-1.7B/      # Pre-downloaded model