English | 中文

🌌 LunaVox: High-Performance C++ Inference Engine for Qwen3-TTS

LunaVox is a high-performance C++ inference engine built specifically for Qwen3-TTS. A streamlined architecture and deep hardware optimization deliver stable, low-latency TTS for embedded devices, desktop apps, and servers alike.

🚀 Key Features

• Lightweight runtime — ONNX Runtime + a custom llama.cpp wrapper, no heavy Python required at inference time.
• Native multi-language — automatic language detection across Chinese, English, Japanese, Korean, Russian, German, French, Italian, Spanish, Portuguese.
• Unified Voice API — one engine.synthesize(text, voice, params) covers Base, Voice Cloning, Custom Voice, and Voice Design.
• HTTP + WebSocket serving (lunavox serve): FastAPI app with POST /v1/synth and streaming WS /v1/stream — see serve guide.
• Desktop GUI (lunavox gui): customtkinter app (Synthesize / Library / Settings) driving the same in-process engine.
• Profile-driven CLI — layered ~/.lunavox/config.toml / env / flag precedence so lunavox --profile quality synth … is a one-liner.
• Cross-platform hardware acceleration — CUDA (NVIDIA), CoreML/Metal (Apple), DML (DirectX 12), and Vulkan.

🛠️ Requirements

• Windows 10/11 (VS 2022/2025), Linux Ubuntu 22.04+ (GCC ≥ 9.0), or macOS 12+ on Apple Silicon
• CMake 3.16+ (Ninja recommended) and a compatible C++17 compiler
• Python 3.10+ for the CLI and conversion toolchain

📊 Performance

Model Qwen3-TTS-12Hz-0.6B-Base with ref/ref_0.6B.json cloning on Intel i9-12900K + RTX 3090 / Windows 11 — 5 warmup + 100 measurement runs on a fixed 25-word English sentence. Full per-run distribution in benchmark/report.md; detailed analysis in Windows performance report.

📦 Install & quick start

pip install lunavox               # core CLI + GUI + HTTP/WebSocket server (default)
pip install "lunavox[convert]"    # + source → GGUF conversion toolchain (heavy, optional)

lunavox bootstrap                 # one-key: pull model + libs + build + smoke test

Prefer step-by-step? Run lunavox model pull, lunavox build libs, then lunavox build --clean. For CUDA see CUDA on Windows. Full command reference: CLI manual.

🎙️ Synthesis

lunavox synth "Hello from LunaVox." -o out.wav                       # base voice
lunavox synth "…" --voice clone  --ref ref/ref_0.6B.json -o out.wav  # voice cloning
lunavox synth "…" --voice custom --speaker Vivian --instruct "…"     # catalog speaker
lunavox synth "…" --voice design --instruct "A warm, calm narrator." # text-to-voice design
lunavox gui                                                          # desktop GUI

The standalone ./build/lunavox-cli works the same way in Python-free environments. Full mode documentation: usage tutorial.

Embedded Python

from lunavox.runtime import Engine, SynthesisParams, Voice

with Engine("models/base_small") as engine:
    result = engine.synthesize(
        "Hello from LunaVox.",
        voice=Voice.clone_file("ref/ref_0.6B.json"),
        params=SynthesisParams(temperature=0.7),
    )
    print(f"RTF {result.stats.rtf:.3f}")  # result.audio is a float32 [-1, 1] mono array

📈 Observability

• --stats-json report.json — RTF + memory breakdown per synthesis
• logs/latest.log — build and runtime output
• -j N — CPU thread count (default 4)

📖 Documentation

Full bilingual docs — guide, CLI reference, technical details, benchmarks, Python API — are published at https://lux-luna.github.io/LunaVox/. Local preview:

pip install -e ".[dev]"
mkdocs serve

Release history: CHANGELOG.md.

🙏 Acknowledgements

• Qwen3-TTS — base weights and architecture
• onnxruntime — audio decoding backend
• llama.cpp — LLM sequence prediction core