Skip to main content

covtl-synth — articulatory synthesis from text (COVTL + VocalTractLab)

PyPI version

(PyPI distribution covtl-synth; the Python module and the CLI stay vtl_synth / vtl-synth.)

Version 1.2.1 hardens the 1.2.0 release: the whole-phrase phonation selector (tag_overrides={'*': {'voicing': ...}} — one entry drives breathy / creaky / whisper over every segment; an explicit per-key entry wins), the full-path phonation checks now collected by CI (tests/test_phonation_full_path.py, 519 green tests), and corrected packaging metadata (description names the headline features, Python 3.13/3.14 classifiers). The 1.2.0 foundations remain: phonation modes audible end-to-end through Pipeline(tag_overrides=...), the covtl-sandhi rule package behind the --sandhi hook, and a coverage campaign in five waves (74.5 % → 76.7 %, honest ceiling 79.0 %; see the coverage dossier).

Version 1.1.0 added the multi-speaker registry, the 6-language pack (de/en/es/fr/it/pt), expressive prosody and the polar-coordinate video — engine outputs unchanged (bit-identical baselines).

covtl-synth (module vtl_synth) turns English text (or phonetic input) into:

  • a .tract file — the 400 Hz sequence of 19 tract + 11 glottis VocalTractLab parameters,
  • a .wav waveform (44.1 kHz, synthesized by VocalTractLab 2.4),
  • a .mp4 video of the sagittal cross-section of the vocal tract, synchronized with the audio — and, with --polar, a dual-panel video adding the live (ρ, θ) polar plane (vocalic branch in red, consonantal in blue).

The synthesis engine is the Berthommier articulatory model (COVTL): place of articulation is computed in polar coordinates (ρ, θ) with coarticulation, then projected onto the VocalTractLab parameters; a tag-based extension engine drives voicing, nasality, laterality, glottal effort and F0 (timed envelopes indexed by the segment events); a syllabic trajectory engine (syltraj) shapes the consonant/vowel arcs, inter-word pauses and phrase-final holds.

The package layout follows the vtl-pipeline reference repository (CLI, package data, video encoder, g2p).

Repository: https://github.com/fberthommier/covtl-pipeline

Documentation

  • Documentation index — docs/README.md: the five manuals (LaTeX sources + compiled PDFs, English US).
  • Reference manual — docs/manual.pdf: full model description with the polar-coordinate phonetic targets ($\rho, \theta$ plane, projection coefficients) and the coarticulation mechanisms (selector superposition, S-curve arcs, cluster scoping), CLI/API reference and worked example.
  • Speakers — docs/speakers.pdf: the multi-speaker registry, install_speaker.py, the four layers of a switch, provenance chains and the defect catalogue (v1.1.0).
  • Language pack --- docs/language_pack.pdf, covering the six languages, setup_lang.py and the marker-only LANG SECTION (v1.2.0): the language selects pronunciation modules, lexicons, G2P and notation, while vowel targets stay the active speaker’s own (the former per-language overrides and JD3 guard were removed, defect D24). Pack internals: lang_pack/README.md, lang_pack/manual/manual.pdf, lang_pack/ARCHITECTURE.md. Speaker characteristics: docs/SPEAKERS.md.
  • Polar visualization — docs/polar_visualization.pdf: the (ρ, θ) plane, the dissociated vocalic/consonantal branches and the dual-panel video (v1.1.0).
  • Interactive HTML demos (online) — https://fberthommier.github.io/covtl-pipeline/html/index.html: the dual-panel polar view as interactive web pages, served by GitHub Pages and rendered straight in the browser (audio-synchronized playback, scrubbing, slow motion, ρ/θ traces) — see Polar video.
  • Expressive prosody — docs/expressive_prosody.pdf: breathing groups, F0 contour, out-of-band stress, and the interaction with the multi-speakers installation (v1.1.0).
  • Inter-word sandhi (--sandhi) — cross-word phenomena (fr liaison, en a/an + linking-r, it raddoppiamento, es enlace, pt encadeamento, de Glottalversatz) via the covtl-sandhi rule package behind the built-in hook: off by default (bit-identical outputs), coverage levels minimal/standard/careful. Since 1.2.0 the full covtl-sandhi rule package (Branch B) ships inside the wheel: --sandhi standard works out of the box; the identity-stub fallback disappears. See docs/ARCHITECTURE.md §8.5.
  • Coverage dossier (waves 1–4) — COVERAGE_REPORT.md + addendum: the four-wave measurement campaign (methodology, per-commit evidence, non-regression proofs), and the wave-4bis addendum documenting the phonation redo (the glottal shape-event channel, measured audio deltas on the four modes). PDF synthesis: docs/wave4/synthesis/COVERAGE_SYNTHESIS_WAVE1-4.pdf. The wave-5 addendum documents the post-1.2.0 hardening audit: the CI-visible phonation tests, the wildcard tag override, and the console alignment (519 green tests, 76.7 % combined, honest ceiling 79.0 %).
  • Architecture — docs/ARCHITECTURE.md: developer map of the engine (module layout, data flow, the B(t) + E_source(t) + E_ortho(t) decomposition) — including the v1.1.0 add-ons (speakers registry, LANG SECTION, polar video).
  • Installation (Windows) — INSTALLATION.md: step-by-step CMD guide (Download ZIP, pip --user, PATH, Python 3.9 note, troubleshooting); see also INSTALL.md.
  • Coding conventions — docs/CONVENTIONS.md.
  • v1.1.0 packaging notes — MIGRATION_NOTES.md: version decision, excluded tests, licensing map.

Quick start

pip install covtl        # the whole suite (engine + console + sandhi)
# or the engine alone:  pip install covtl-synth
# from source:          pip install .   (or: pip install -e .)
vtl-synth run "this is easy for us" -o ./out

Output: out/this_is_easy_for_us.tract, .wav, .mp4 and .txt (the SAMPA transcript produced by the g2p). The default speaker is jd3; see Multi-speakers to install another voice, and Languages for de/es/fr/it/pt input.

Multi-speakers

Five speakers ship in the registry vtl_synth/data/speakers/ and are selected with the reversible tool install_speaker.py (constants + orthogonal branch + wheel resource; backups and rollback automatic, smoke-tested in a fresh subprocess):

name origin f0 default
jd3 reference (SynthVTL24b original) 102.2 Hz
s1 DVTD MRI subject 1 (male), calibrated constants v3 114.9 Hz
s2 DVTD MRI subject 2 (female), calibrated constants v3 181.0 Hz
m01 official VTL 2.4 ZIP (male), pure option A + 2025 glottal mapping 120.0 Hz
w02 official VTL 2.4 ZIP (female), pure option A + 2025 glottal mapping 232.5 Hz
python install_speaker.py list            :: registered + active
python install_speaker.py install m01     :: swap speaker (reversible)
python install_speaker.py status          :: hashes / coherences
python install_speaker.py restore         :: back to jd3

Notes: the shared vocaltractlab_cython wheel resource (audio + SVG layer) is swapped for the next process — relaunch your program after an install. Per-speaker regression baselines (regression_baselines_<name>.json) are picked automatically by the test suite; regenerate them with scripts/regen_baselines.py after an intentional signal change.

All five speakers validated in CI

The regression baselines are speaker-relative, so CI runs them for every registered speaker, not just the active one: scripts/run_baselines_matrix.py swaps the two repository layers the tract/glottis signals depend on (core/constants.py + data/speakers/ACTIVE_SPEAKER) and runs tests/test_regression_baselines.py in a fresh subprocess per speaker (constants are imported at module load — an in-process swap would be poisoned by sys.modules). The repository state is restored bit-exactly on exit, whatever happened. The same script regenerates all five baseline files (--regen) after an intentional signal change. On GitHub Actions this is the baselines job of ci.yml (windows-latest — the baselines are generated and acoustically validated on Windows); locally:

python scripts/run_baselines_matrix.py              # test all 5
python scripts/run_baselines_matrix.py --speaker s2 # test one
python scripts/run_baselines_matrix.py --regen      # regen all 5

Each of the five baseline files pins, per phrase of the corpus, the tract and glottis matrices: per-column min/max/mean statistics plus tract_sha256 / glottis_sha256 over the quantized (1e-6) signal — the glottal layer is pinned since schema 3, so the phonation chain itself is regression-guarded (see COVERAGE_REPORT.md).

Languages

setup_lang.py + lang_pack/ add German, English, Spanish, French, Italian and Portuguese (g2p, Wikipron lexicons, per-language vowel sets and prosody profiles). The active language is a managed LANG SECTION inside constants.py, transported across speaker swaps and fully reversible:

python setup_lang.py list                 :: available + current
python setup_lang.py install --lang fr    :: install/activate French
python setup_lang.py setlang de           :: switch language
python setup_lang.py calibrate fr         :: re-derive vowel targets (JD3 only)
python setup_lang.py verify               :: integrity check
python setup_lang.py restore              :: back to native vowels

Guard-rail: the pack vowel targets are calibrated in situ on JD3 — calibrate (re)installs speaker jd3 first and says so; the other speakers keep their native vowel targets (a non-jd3 speaker with an active language is flagged as an unverified combination). Lexicons are CC BY-SA 3.0 (Wikipron) — keep the README_xx_lexicon.md notices beside the TSV files; details in lang_pack/LICENSE.md.

Polar video

vtl-synth run --polar "this is easy for us" -o ./out also writes a .polar intermediate (100 Hz, format v4: dissociated vocalic / consonantal trajectories) and renders a dual-panel MP4 — sagittal cross-section (left) and polar (ρ, θ) plane (right) at 25 Hz, vowels in red, consonants in blue, each branch with a fading trail. See docs/polar_visualization.pdf and the short demo examples/polar_demo_en.mp4.

Interactive HTML demos (docs/html/): the same dual-panel view as self-contained web pages — play/pause, scrub, slow motion, ρ/θ traces and the synthesized audio are synchronized (no server, no network). Online on GitHub Pages: index · FR « ba.da.ga » · EN « this is easy for us »; from a clone, open docs/html/index.html directly. The pages are regenerated by python docs/html/make_html_demos.py.

Expressive prosody

vtl-synth run --expressive "…" -o ./out (default off, outputs bit-identical without it) adds syntactic breathing groups (per-language function-word dictionaries, hard 5-word balance, inseparable connectors, 130 ms % breath pauses) and a sculpted F0 contour (declination + block re-attacks + pitch accents + per-language nuclear fall; lexical stress travels out-of-band, never in the SAMPA). It only touches the glottis f0 column and pause durations — never the tract, so it composes with any speaker. Demo: python examples/demo_prosody.py; measurements: python scripts/f0_report.py <tract…>. Guide: docs/expressive_prosody.pdf.

Command line

vtl-synth run "this is easy for us" -o ./out      # tract + wav + mp4
vtl-synth run --phonetic "ba da ga | iowa a g" -o ./out   # direct SAMPA
vtl-synth run -s m01 -l fr "bonjour" -o ./out     # speaker + language
vtl-synth run --expressive "this is easy for us" -o ./out
vtl-synth text-to-wav "hello world" -o ./out      # audio only
vtl-synth tract-to-mp4 out/x.tract out/x.wav -o out/x.mp4
vtl-synth inspect out/x.tract --states 2 --ranges
vtl-synth plot-tract out/x.tract --which all -o out/x.png
vtl-synth polar-build out/x.tract out/x.wav -o out/x.polar
vtl-synth polar-video out/x.polar out/x.tract out/x.wav -o out/x.mp4

Speaker/language options (-s/--speaker, -l/--lang, --expressive/--no-expressive) act for the run: they switch the registry entry / language section first (same machinery as install_speaker.py / setup_lang.py). The timing defaults shown as None in --help (100/80/200/320 ms) are resolved through the active language profile.

plot-tract is the graphical counterpart of inspect: it draws the 400 Hz trajectory of each tract parameter (optionally the glottis f0 and pressure with --which all, or a subset with --params JX JA TTX TTY) into a PNG figure; --ranges adds the min/max/mean table of inspect. Requires the optional matplotlib dependency (pip install .[plot]).

Engine options (COVTL model parameters) on run/text-to-wav:

option default description
--tcons MS 100 consonant gesture duration
--tvoy MS 80 vowel gesture duration
--tpause MS 200 short inter-word pause (space)
--tpause-long MS 320 long pause (end of sentence, |)
-f0 HZ per speaker F0 (jd3 default 102.216 Hz); a French-style declination is applied
--no-ortho off disable the orthogonal branch (TS3/VS)
--engine syl trajectory engine (syl recommended, legacy = pre-syltraj)
--fps, --scale, --no-video 25, 2 video rendering

Phonation modes are a Python-API feature (tag overrides on the converted vowel keys); the CLI stays phonetically neutral by default.

Note on the arc-shape constants (see syltraj.py and the CHANGELOG): the syl engine uses the convention ν = −1, K = 1000, SYL_COEFCEN = 0.5 — ν = 1 is recovered when the cosine is written with the opposite sign (the published equation's form); the reference curvatures K = 10/30 with ν = 1 belong to the legacy engine and the polar-video display.

Python API

from vtl_synth import Pipeline

pipe = Pipeline()                       # validated default settings
result = pipe.run("this is easy for us", output_dir="./out")
print(result.sampa)                     # "Dis.iz.i.zi.fOR.@s"
print(result.wav_path, result.duration_s)

# phonetic input, audio only
result = pipe.text_to_wav("ba da ga | iowa a g",
                          output_dir="./out/phon", use_g2p=False)

# phonation modes: breathy / creaky / whisper on the vowels
# (level-C tag overrides, converted segment keys)
pipe_breathy = Pipeline(tag_overrides={
    '@': {'voicing': 'breathy'}, 'i': {'voicing': 'breathy'}})
result = pipe_breathy.run("this is easy for us", output_dir="./out")
# the 4 modes produce 4 distinct WAVs (measured RMS delta vs normal:
# breathy 0.017, creaky 0.048, whisper 0.026)

# whole-phrase phonation in one entry (since wave 5): the special
# key '*' overrides every segment; an explicit per-key entry wins
pipe_whisper = Pipeline(tag_overrides={'*': {'voicing': 'whisper'}})

# the four modes side by side: python examples/demo_phonation.py

Audio demos of the four modes: docs/demos/ (normal / breathy / creaky / whisper, RMS measured).

The full engine API (72 symbols: build_phrase_tract, text_to_tract, timers, envelopes, polar model, …) is available in vtl_synth.core; the VTL bindings in vtl_synth.vtl.api.

Input notation

  • English text — converted to phonemes by the bundled g2p (CMUdict, ARPAbet → engine SAMPA). Words are connected with '.' inside a phrase (syllabified by onset maximization), so the engine coarticulates across word boundaries — no pause inside a phrase: this is easy for us → Dis.iz.i.zi.fOR.@s. A comma keeps a short inter-word pause (space) and sentence punctuation becomes | (long pause). Unknown words are passed through verbatim when they are already valid SAMPA.
  • SAMPA phonetic input (--phonetic) — same engine notation, written directly: vowels a e i o u y E O @ 2 6 9, consonants p b t d k g f v s z S Z T D tS dZ m n J l R j w h ɥ; . syllable break, space short pause, | long pause.

Repository layout

covtl-pipeline/
├── pyproject.toml            # package config (Python ≥ 3.9)
├── launch.bat                # Windows quick-launch script
├── install_speaker.py        # multi-speakers selector (add-on 1)
├── setup_lang.py             # language pack installer (add-on 2)
├── lang_pack/                # language pack source (g2p, lexicons, manual)
├── vtl_synth/
│   ├── core/                 # synthesis engine
│   │   ├── pipeline.py       # Pipeline orchestrator (run/text_to_wav)
│   │   ├── phonemes.py       # ARPAbet↔SAMPA tables
│   │   ├── speaker_registry.py  # multi-speakers registry access
│   │   └── … 27 modules incl. __init__ (constants, polar, syltraj, timers, …)
│   ├── cli/main.py           # vtl-synth entry point
│   ├── video/                # SVG/PNG renderer + MP4 encoder + polar
│   │                         # video (dual panel) + tract figure
│   ├── vtl/api.py            # VocalTractLab API facade
│   ├── utils/                # g2p (CMUdict) + language profiles
│   │                         # (chunking, prosody_f0, … — add-ons 2-3)
│   └── data/
│       ├── vtl_binaries/JD3.speaker     # legacy fallback speaker file
│       ├── speakers/                    # registry: 5 .speaker +
│       │                                # *_constants.py + registry.json
│       └── *_lexicon.tsv                # installed Wikipron lexicons
├── scripts/                  # regen_baselines, calibrate_fine_v2, f0_report…
├── tests/                    # lightweight suite (153 tests, engine
│                             # non-regression + CLI/g2p + registry)
├── examples/                 # demo.py, demo_prosody.py,
│                             # polar_demo_en.mp4, html/ interactive demos
├── docs/                     # manual, ARCHITECTURE, CONVENTIONS,
│                             # expressive_prosody.pdf, polar_visualization.pdf
├── regression_baselines*.json   # E2E non-regression baselines (5 speakers)
└── MIGRATION_NOTES.md        # v1.1.0 packaging decisions & inventory

Differences with the vtl-pipeline reference repository

The reference repository chains VTL-native stages (text → .seg → .ges → tract → wav/mp4) through a ctypes binding to libVocalTractLabApi. This package instead synthesizes the 400 Hz tract sequence directly with the Berthommier COVTL model — there is no .seg/.ges stage, hence no such files in the output; the VTL engine is the vocaltractlab-cython wheel (no manual DLL management).

Installation

Windows: see INSTALLATION.md (step-by-step CMD guide, pip-only, Git optional). Short version: pip install . — dependencies numpy, scipy, vocaltractlab-cython, Pillow, imageio-ffmpeg (bundled ffmpeg), cmudict. Optional extra plot (matplotlib) for plot-tract.

License

GPL-3.0-or-later — see LICENSE and THIRD_PARTY_NOTICES.md. This licensing is mandatory: the three core components (VocalTractLab, vocaltractlab-cython, create_vtl_corpus concepts) are GPL-3.0, so this package may not be relicensed under a more permissive license (MIT, Apache-2.0, BSD).

Copyright (C) 2026 Frédéric Berthommier. The synthesis engine (vtl_synth/core) implements the Berthommier COVTL articulatory model coupled with VocalTractLab, published under GPL-3.0-or-later. When redistributing, keep the full LICENSE file and THIRD_PARTY_NOTICES.md with all copyright notices intact.

References

If you use this package in academic work, please cite the publications of the Berthommier COVTL model:

  1. F. Berthommier, "A mathematical model of the vowel space", arXiv:2111.00868 [eess.AS], 2021. DOI: 10.48550/arXiv.2111.00868 — https://arxiv.org/abs/2111.00868
  2. F. Berthommier, "Why can big.bi be changed to bi.gbi? A mathematical model of syllabification and articulatory synthesis", arXiv:2307.02299 [eess.AS], 2023. DOI: 10.48550/arXiv.2307.02299 — https://arxiv.org/abs/2307.02299
  3. F. Berthommier, "Synthèse de syllabes avec un modèle de Maeda piloté par une représentation complexe", in Actes des 35èmes Journées d'Études sur la Parole (JEP), pages 541–550, ATALA and AFPC, 2024. https://aclanthology.org/2024.jeptalnrecital-jep.55/
  4. F. Berthommier, "Articulatory modeling of the S-shaped F2 trajectories observed in Öhman's spectrographic analysis of VCV syllables", Interspeech 2025, Rotterdam, Netherlands. HAL: hal-05233039, DOI: 10.5281/zenodo.17018832 — https://hal.science/hal-05233039v1

Please also cite VocalTractLab itself (see THIRD_PARTY_NOTICES.md).

Metadata

Release files for covtl-synth 1.2.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for covtl-synth 1.2.2
File Size Uploaded Provenance Actions
covtl_synth-1.2.2.tar.gz 3.4 MB Without provenance Details for covtl_synth-1.2.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for covtl-synth 1.2.2
File Interpreter ABI Platform Actions
covtl_synth-1.2.2-py3-none-any.whl Python 3 none any Details for covtl_synth-1.2.2-py3-none-any.whl

Total release size: 6.8 MB

Release files / covtl_synth-1.2.2.tar.gz

Download URL covtl_synth-1.2.2.tar.gz
Size 3.4 MB
Tags Source
SHA-256 checksum
How to use checksums
ad6a3cbe550ab13f8f970e1b38cd697b6b1f2b7091157933f599eacb93a7c2f4
BLAKE2b-256 checksum
How to use checksums
a15da7ea8b7cbe937285c8101dd7a1fca79c84096db7d389f6930b218d72b792
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.10

Release files / covtl_synth-1.2.2-py3-none-any.whl

Download URL covtl_synth-1.2.2-py3-none-any.whl
Size 3.4 MB
Tags Python 3
SHA-256 checksum
How to use checksums
27a8f87c90dfd6b871dc256657825df8ed84d1311c54d6be2bc07e39b4ca3030
BLAKE2b-256 checksum
How to use checksums
355c7becfebfffe68c913e6965478251e760c6d4e6e7d7ebe069aa106c1bc9b1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.10

Security for covtl-synth 1.2.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.2.2 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page