thalovant-languages
Every word a Thalovant component's rule turns on, for every language the sources know, derived from those sources and kept in one place, out of every codebase.
A voice satellite waits on "turn off the" because "the" is a continuation word. An intent listing closes "quelle heure est-il" with a question mark because "quelle" opens a question, and reads "volume {level} percent" aloud as "volume cinquante pour cent" in French. A French synthesiser is told to say "onze minutes" where it would swallow a consonant. Each of those is a fact about a language, not about the program that needs it. Here they are data: 292 language and locale directories, read by the SDK and by the satellite, and none of it written by hand.
Where the words come from
| What | Source | How |
|---|---|---|
continuation_words |
Universal Dependencies treebanks (150 languages) | a word that is a determiner, preposition, conjunction, auxiliary or possessive nearly every time it appears, and almost never the last word of a sentence |
trailing_words |
Universal Dependencies | the continuation words that are neither a possessive nor an auxiliary: "coupe le son" and "qué hora es" are whole sentences, not prefixes |
question_openers |
Universal Dependencies | an interrogative word (PronType=Int), or a word a sentence usually is a question when it starts with it |
question_words_anywhere |
Universal Dependencies | an interrogative word a sentence almost always is a question when it holds it, wherever it sits |
written_forms |
Universal Dependencies | a closed-class word written capitalised mid-sentence nine times in ten (i: I) |
plural |
Unicode CLDR (190 locales) | the language's plural rules, verbatim, evaluated at runtime by _plural.py |
lowercase_map |
Unicode SpecialCasing.txt |
a language's own unconditional lower-case mappings: Turkish and Azeri's dotted and dotless i |
scripts.yaml |
Unicode PropList.txt, Scripts.txt |
scripts written without spaces, scripts where a character is a syllable, the marks that close a sentence (Sentence_Terminal) or break a clause (Terminal_Punctuation), spaced or unspaced |
slot_examples, speech_substitutions, question_patterns |
overrides/<tag>.yaml |
what only a person knows: how a slot reads aloud in a house, what a Piper voice mispronounces, an inversion no treebank lists |
Every generated file starts with a header naming its sources, their versions, the treebanks it was read from, and the thresholds that decided what got in. The thresholds are numbers a reader can argue with; the argument then has its facts. Two of them were settled by measurement on the voice satellite: English "is" ends three sentences in a hundred and is worth waiting on, "on" ends six ("turn it on") and is not.
Layout
src/thalovant_languages/languages/
scripts.yaml what a writing system does that no word list can
en/language.yaml English, every region
fr/language.yaml French
pt/language.yaml Portuguese ...
pt-PT/language.yaml ... and what European Portuguese does differently
overrides/
en.yaml, fr.yaml what only a person knows, laid on top
scripts.yaml the two marks the synthesiser splits on that Unicode does not list
scripts/derive.py the generator
A directory is named by a BCP-47 tag: bare when what it says holds for
every region, regional when a source says the region differs (CLDR gives
pt-PT its own plural rule). A regional file carries only the difference;
the loader lays it over the language's file.
Use
import thalovant_languages as languages
languages.language("fr-CA")["continuation_words"] # the French file
languages.words("en", "trailing_words") # lower-cased, as a set
languages.plural_category("ru", 21) # "one", by CLDR's rules
languages.asks("on mange à quelle heure ce soir", "fr") # True: it holds a question word
languages.language("tlh") # {} -- nothing describes it
languages.script_pattern("unspaced").search("今天") # a character of an unspaced script
languages.marks("sentence_ends", "spaced") # every Sentence_Terminal mark, and "…;"
A language is found by the matcher the rest of OVOS uses
(ovos_spec_tools.language): fr-CA reads fr, en-GB reads en, and a
language nothing describes gets an empty mapping rather than another
language's rules. The keys a component reads are thalovant_languages.KEYS.
Since 0.3.1, Traditional Chinese tags (zh-TW, zh-HK, zh-MO,
zh-Hant) find a zh-Hant word list. Simplified Chinese tags can use
zh-Hans. Both inherit shared Chinese rules. These are writing-system
variants, not Cantonese translations. asks("什麼時候出發", "zh-TW") works
without inserting spaces or a question mark.
SkillKit owns skill dialog translations and regional resource generation. This package supplies shared language rules; it does not need a copy of every skill's regional folders.
THALOVANT_LANGUAGES_DIR=/path/to/checkout/src/thalovant_languages/languages
reads a checkout instead of the installed data, for trying a change before
it is released; languages.refresh() forgets what was read after changing it.
Regenerating
pip install -e ".[dev]"
curl -LO https://www.unicode.org/Public/UCD/latest/ucd/PropList.txt # and Scripts.txt, SpecialCasing.txt
curl -LO https://raw.githubusercontent.com/unicode-org/cldr-json/main/cldr-json/cldr-core/supplemental/plurals.json
# the treebanks: https://universaldependencies.org/ -> the current release archive (~700 MB)
python scripts/derive.py --ud ud-treebanks-v2.18 --cldr plurals.json --unicode .
thalovant-languages check
git diff --stat
The generator reads about four gigabytes of treebanks in a minute or two
and rewrites every file; review the diff, commit. To change a word, do not
edit the file: change the generator's thresholds, or put what you know in
overrides/, and rerun. thalovant-languages check refuses a key no
component reads, a pattern that does not compile, a plural rule the
evaluator cannot read, and a list holding a boolean.
The Chinese treebanks contain both writing systems. After deriving zh,
the generator uses OpenCC 1.4.2 to produce deduplicated zh-Hans and
zh-Hant word lists. OpenCC is only needed for development; it is not a
runtime dependency. To regenerate those lists from the committed Chinese
source, without downloading treebanks again:
python scripts/derive_chinese.py
python scripts/derive_chinese.py --check
The check fails when a generated list is missing or stale. Tests run it too.
Install and check a release
python -m pip install --upgrade thalovant-languages
thalovant-languages check
PyPI receives releases first. Piwheels publishes Raspberry Pi wheels independently, so a new release may appear there later. The package itself is pure Python; its wheel also works across platforms.
Who reads it
thalovant(the Python SDK):thalovant.listingsets a registered pattern the way a person reads it, ranks phrases, and reads slots aloud.thalovant-voice(the satellite): the semantic endpoint's continuation words, the lower-casing of an all-capitals transcript, the plural forms of its own counted text, the synthesiser's pronunciation repairs.- the fleet's fallback skills (weather, date and time): a keyword net claims
a sentence only when
asks()says it is a question.
Nothing here parses dates, numbers or colours; the OVOS parsers do that for
skills on the hub. Contractions come from ovos-utterance-normalizer, which
already carries them.
Metadata
Release files for thalovant-languages 0.3.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| thalovant_languages-0.3.6.tar.gz | 87.7 kB | Without provenance | Details for thalovant_languages-0.3.6.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| thalovant_languages-0.3.6-py3-none-any.whl | Python 3 | none | any | Details for thalovant_languages-0.3.6-py3-none-any.whl |
Total release size: 328.4 kB
Release files / thalovant_languages-0.3.6.tar.gz
| Download URL | thalovant_languages-0.3.6.tar.gz |
|---|---|
| Size | 87.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9ad350ead7e9563d98c88ed967fa226ae6dbfcab7b918e9d9f6376723063b18d
|
|
BLAKE2b-256 checksum How to use checksums |
acae9d99a017ff25ce255a9a550a9f201372662600951866221abb61bd13e561
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / thalovant_languages-0.3.6-py3-none-any.whl
| Download URL | thalovant_languages-0.3.6-py3-none-any.whl |
|---|---|
| Size | 240.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3ad7a209d182d22815210f3a998e591bd9bbe36eb432e87ca859014a7c01c5fa
|
|
BLAKE2b-256 checksum How to use checksums |
ab4cc3f67844109ef0c5940ba1dabf8784fffe51ed0595519ab99939af633be4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Security for thalovant-languages 0.3.6
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.