qwentts-cpp-python

Python bindings and wheel packaging for Pascal's qwentts.cpp C ABI.

This package is intentionally small:

• it loads libqwen with ctypes
• it exposes buffered and streaming synthesis
• it can bundle prebuilt libqwen/libggml binaries in platform wheels
• it does not bundle GGUF model weights

CUDA development build with an existing qwentts.cpp checkout:

python scripts/build_native.py \
  --source /path/to/qwentts.cpp \
  --backend cuda \
  --clean
QWENTTS_CPP_WHEEL_BUILD_TAG=1cu128 python -m build --wheel

CPU development build:

python scripts/build_native.py \
  --source /path/to/qwentts.cpp \
  --backend cpu \
  --clean
QWENTTS_CPP_WHEEL_BUILD_TAG=1cpu python -m build --wheel

--backend metal is the default on macOS; other platforms default to CUDA. CPU builds are still useful for development and smoke tests.

Installation

Starting with 0.4.1, PyPI provides Linux CUDA 12.8 wheels and macOS 14+ Apple Silicon Metal wheels. Pip selects the appropriate wheel automatically:

pip install qwentts-cpp-python

Additional backend-specific wheels are published to Hugging Face Hub as local-version variants. Use them when the PyPI CUDA 12.8 wheel does not match the runtime or GPU target, for example DGX Spark / GB10 with CUDA 13:

pip install "qwentts-cpp-python==0.4.0+cpu" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cpu

pip install "qwentts-cpp-python==0.4.0+cu124" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu124

pip install "qwentts-cpp-python==0.4.0+cu128" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu128

pip install "qwentts-cpp-python==0.4.0+cu130" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130

These commands use pip's --find-links mode against the Hugging Face directory page for the selected flavor. Dependencies still resolve from PyPI normally. The wheels do not bundle CUDA runtime or cuBLAS libraries; use a base image or system installation that provides the matching CUDA runtime.

The Hugging Face wheel pages may contain multiple Linux compatibility tags for the same backend flavor. For example, the cu128 page can host both manylinux_2_35 wheels for Ubuntu 22.04+ and manylinux_2_39 wheels for Ubuntu 24.04+. Pip selects the newest compatible wheel for the current machine.

Pull requests do not build the Linux wheel matrix. The PyPI and Hugging Face publishing workflows each rebuild fresh wheels from the pinned qwentts.cpp revision; validation artifacts are not reused for publishing.

Apple Silicon (Metal)

For macOS 14 or newer, arm64 Python 3.10+, install directly from PyPI:

python -m pip install --only-binary=qwentts-cpp-python "qwentts-cpp-python>=0.4.1"
python -c "from qwentts_cpp import QwenLibrary; print(QwenLibrary().version())"

No local CMake build, Homebrew libraries, or explicit library_path is needed. The wheel bundles libqwen and the ggml CPU, Metal, BLAS, and core libraries; Metal shader source is embedded in the Metal library. Apple frameworks come from macOS. The build applies a shader-only workaround for the pinned ggml's invalid scalar-to-BF16-vector fill cast, restoring the source checkout afterward; the C ABI is unchanged. Use a native arm64 Python, rather than an Intel Python under Rosetta. Downstream packages can declare qwentts-cpp-python>=0.4.1 as a regular dependency, without an extra index or a direct wheel URL. The macOS platform tag distinguishes Metal wheels from Linux CUDA wheels on PyPI. Intel Macs and macOS versions older than 14 are not supported by this Metal wheel.

The earlier 0.4.0 Metal wheel remains available on Hugging Face:

python -m pip install "qwentts-cpp-python==0.4.0+metal" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/metal

The +metal version identifies the backend on Hugging Face; PyPI wheels use the plain public version without this suffix.

Before publication, download the hf-wheel-metal-macosx-arm64 artifact from the PR's Apple Silicon Metal wheel check, unzip it, and install its .whl with python -m pip install /path/to/qwentts_cpp_python-0.4.1+metal-*.whl. This macOS check also runs for PRs; the Linux matrix remains dispatch-only. Both publishers rebuild and check their own Metal wheels.

Run a streaming smoke test using local GGUF weights (weights are not bundled):

python scripts/smoke_stream.py \
  --talker /path/to/qwen-talker-1.7b-base-Q8_0.gguf \
  --codec /path/to/qwen-tokenizer-12hz-Q8_0.gguf \
  --ref-spk /path/to/reference.spk \
  --require-metal --output metal-smoke.wav

For a CustomVoice talker, replace --ref-spk with --speaker Vivian; for a VoiceDesign talker, use --instruct "A calm, clear voice.". --require-metal forces the Metal device so the test fails if only CPU execution is available. The script checks multiple nonempty, finite audio chunks and writes a mono 24 kHz PCM WAV. Listen to it with afplay metal-smoke.wav. Hosted macOS CI checks packaging, ABI layouts, and installed loading; synthesis requires a Mac with a working Metal device and local weights.

To build the same wheel locally with the pinned source checkout:

python -m pip install build delocate twine
export MACOSX_DEPLOYMENT_TARGET=14.0
python scripts/set_local_version.py metal  # changes local version metadata
python scripts/build_native.py --backend metal --clean
python -m build --wheel --config-setting=--build-option=--plat-name=macosx_14_0_arm64
delocate-wheel --require-archs arm64 -w wheelhouse -v dist/*.whl
python -m twine check --strict wheelhouse/*.whl

Native ABI compatibility

The CI wheel build defaults to qwentts.cpp 6fae92914045cd83364d2845ceaa0f7969727319 (September 28, 2026), which requires ABI v5 and includes the upstream batched compute worker, codec memory controls, language/model queries, and latest ggml fork update.

ABI v5 changes the initialization and synthesis struct layouts; libraries built from the previous ABI v2 pin must be rebuilt. Python's do_sample=False and subtalker_do_sample=False now select greedy decoding by setting the corresponding native temperature to zero. Buffered codec chunk size is set once with QwenTTS(..., codec_chunk_sec=24.0) or QwenTTS.from_pretrained(..., codec_chunk_sec=24.0), rather than on synthesize(). Zero selects the native default. The codec derives its own left context; codec_left_context_sec is no longer accepted. Streaming's codec_chunk_sec still controls Python output packet size independently. The wrapper continues to serialize requests on each context (native max_batch=1). tts.language_names() lists the model's languages (synthesis also accepts lang="auto"), and tts.model_type() reports base, custom_voice, or voice_design.

The loader verifies this native revision before calling functions that write ctypes parameter buffers. Upstream does not expose an ABI-version or struct-size query, so other revisions (including unknown builds) are rejected with an actionable error even if they may be compatible. When updating the pin, update QWENTTS_NATIVE_REVISION in the binding and run tests/test_native_abi.py with QWENTTS_CPP_SOURCE pointing to the new checkout to verify every struct size and field offset. An incompatible library fails before model loading.

QWENTTS_CPP_WHEEL_BUILD_TAG is useful for local wheelhouses. For public indexes, publish one backend flavor per package/version/platform compatibility tag; otherwise pip has no way to choose between CPU and CUDA binaries.

Local smoke test with a built library:

QWENTTS_CPP_LIBRARY=/path/to/libqwen.so python - <<'PY'
from qwentts_cpp import QwenLibrary
lib = QwenLibrary()
print(lib.version())
PY

Model files are resolved with huggingface-hub by QwenTTS.from_pretrained(...) or passed directly to QwenTTS(...) as GGUF paths.

Native logging in Python applications

Pass log_level="warning" when creating a context to hide routine qwentts.cpp and GGML/Metal messages from model loading and synthesis while keeping warnings and errors on stderr. "quiet" is an alias for "warning"; "info" and "debug" expose diagnostics, and "error" shows errors only.

from qwentts_cpp import QwenTTS

with QwenTTS.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    quant="Q4_K_M",
    log_level="warning",
) as tts:
    audio, sample_rate = tts.synthesize(
        text="The sky is blue today.", speaker="Vivian"
    )

The same log_level argument works with QwenTTS(talker_path, codec_path) and QwenLibrary(...). Call set_log_level("debug") on either object to expose all diagnostics later. Configure the level before creating a QwenTTS to cover model initialization. The level and both native callbacks are process-wide: the most recent setting applies to all contexts, persists after a context closes, and should be changed only when no other thread is running native inference. This setting does not redirect the process stderr file descriptor. Existing set_log_callback() receives filtered qwentts.cpp messages; GGML messages use stderr. The package's wheel builds retain an application-installed GGML callback during first backend initialization; custom native builds must use scripts/build_native.py for the same behavior.

Streaming packet sizes

QwenTTS.stream() defaults to a 4-frame first packet (320 ms of mono 24 kHz audio). Choose first_chunk_frames=1, 2, 4, or 8 for 80, 160, 320, or 640 ms of initial audio. A larger packet gives playback more audio to start with, at the cost of waiting longer before the first yield.

for audio, sample_rate in tts.stream(
    text="The sky is blue today.",
    ref_spk_emb=spk,             # Base model; use speaker= for CustomVoice
    first_chunk_frames=4,
    codec_chunk_sec=0.64,       # Default later packets: 8 frames, independently sized
):
    play_audio(audio, sample_rate)  # Your playback/transport function

Later packets default to codec_chunk_sec=0.64 (8 frames), matching the native steady-state width without adding a larger batching delay after the first packet. Explicit values round to the nearest 80 ms frame, with a one-frame minimum (for example, 1.0 second rounds to 13 frames / 1.04 seconds). The value must be finite and positive. Successful end-of-speech or the token limit flushes any remaining audio as a short packet, including utterances shorter than the requested first packet. Cancellation or errors discard the unfinished packet; closing the iterator requests native cancellation.

Packet assembly happens in Python, using the verified ABI v5 library. The native decoder still emits its fixed 1→2→4→8-frame ramp, then 8-frame chunks. Consequently, first packets of 1, 2, 4, and 8 frames become available after native output has reached 1, 3, 7, and 15 frames respectively (or earlier at end-of-speech). Packet boundaries preserve every PCM sample but do not change native decode scheduling. Later packets may become available together when a native callback spans several packet boundaries; this is not a timed playback scheduler. Buffered synthesize() uses the native codec chunk size configured when the context is created; the native codec derives its left context.

last_stream_profile keeps first_callback_* and callback_count for raw native callbacks. first_packet_ready_ms, first_packet_audio_s, and packet_count describe the assembled Python packets; first_yield_ms measures delivery to the caller. This distinction includes the buffering cost in latency measurements instead of treating the first native callback as audible output.

Benchmark 1, 4, and 8 frames on local hardware (one warm-up, then three measured runs per setting, with rotated order):

python scripts/benchmark_first_chunk.py \
  --talker /path/to/qwen-talker-1.7b-base-Q8_0.gguf \
  --codec /path/to/qwen-tokenizer-12hz-Q8_0.gguf \
  --ref-spk /path/to/reference.spk \
  --require-metal --output /tmp/first-chunk-benchmark.json

For other model types, use --speaker or --instruct instead of --ref-spk. Omit --require-metal on other backends. The report includes native first callback time, first Python packet time/duration, and every packet's size. The benchmark checks first/later packet sizes, finite non-silent audio, and the final tail. Playback smoothness still depends on synthesis speed and the player's buffering policy.

Example measurement on an Apple M3 Pro with Metal, macOS 26.6.2, Python 3.12.13, the Q8_0 1.7B Base talker and Q8_0 codec above, and a cached speaker embedding (three runs per setting after warm-up, medians):

All runs used the default 8-frame later packets (codec_chunk_sec=0.64), preserved the 115-frame utterance, and flushed the final short tail. Four frames provide a middle ground on this machine; these measurements do not guarantee gap-free playback on other hardware. Raw native callbacks remain one frame initially; the larger first packet is assembled by the binding.

Cached voice references

qwentts.cpp can skip reference WAV encoding for Base voice cloning by passing precomputed latents:

• .spk: raw float32 speaker embedding from qwen-codec --talker
• .rvq: packed 11-bit reference codec stream from qwen-codec

The wrapper can create those files in-process from decoded mono float32 audio at 24 kHz:

from qwentts_cpp import QwenTTS

tts = QwenTTS.from_pretrained("Qwen/Qwen3-TTS-12Hz-1.7B-Base", quant="Q4_K_M")

# ref_audio_24k is a 1-D numpy float32 array, already resampled to 24 kHz.
voice_ref = tts.extract_voice_ref(ref_audio_24k)
voice_ref.save("reference.spk", "reference.rvq")

from qwentts_cpp import QwenTTS, load_speaker_embedding

tts = QwenTTS.from_pretrained("Qwen/Qwen3-TTS-12Hz-1.7B-Base", quant="Q4_K_M")

spk = load_speaker_embedding("reference.spk")
audio, sr = tts.synthesize(
    text="The sky is blue today.",
    lang="english",
    ref_spk_emb=spk,
    max_new_tokens=128,
)

For ICL clone mode, load the RVQ matrix with the model's codebook count and also pass the reference transcript:

from qwentts_cpp import load_rvq_codes

rvq = load_rvq_codes("reference.rvq", tts.num_codebooks())
audio, sr = tts.synthesize(
    text="The sky is blue today.",
    lang="english",
    ref_spk_emb=spk,
    ref_codes=rvq,
    ref_text="Transcript of the reference audio.",
)