kreyolib 0.4.0

"Kreyòl merite zouti tou · Kreyòl deserves tools too"

If you like this project, a star ⭐️ would mean a lot :)

Table of Contents generated with DocToc

• Overview / Apèsi
• Installation / Enstalasyon
• Usage / Itilizasyon
  • Alphabet / Alfabèt
  • Normalization
    • Orthography (API)
    • Contractions (API)
    • Diacritics (API)
  • Tokenization
    • Sentence Splitter (API)
    • Word Tokenizer (API)
    • BPE Tokenizer (API)
  • Conversion
    • Number to Text (API)
    • Text to Number (API)
    • Datetime to Text (API)
    • Text to Datetime (API)
  • Advanced Models & Intelligence
    • POS Tagger (API)
• Corpus & Data
  • Stop Words (API)
  • Chat Abbreviations (API)
  • Sample Sentences (API)
• Roadmap & Progress / Plan Travay
  • How People Can Contribute

Overview / Apèsi

Haitian Creole is spoken by millions of people, but it still lacks many of the language resources and tools available for larger languages. This project aims to build an open-source Ayiti NLP ecosystem focused on creating useful Natural Language Processing tools for Haitian Creole.

Most NLP progress has focused on high-resource languages, while Haitian Creole remains underrepresented. Creating better tools for Haitian Creole can help preserve the language, improve accessibility, and allow more Haitian developers and researchers to build AI applications.

Installation / Enstalasyon

pip install kreyolib -U

# If you are going to do text_to_datetime
pip install kreyolib[datetime] -U

# If you are going to do BPE tokenization
pip install kreyolib[bpe] -U

bpe pulls in numpy and sentencepiece, which the rest of the library does not need. It is only required for kreyolib.tokenize.bpe.

Usage / Itilizasyon

Alphabet / Alfabèt

The package exposes the full Haitian Creole alphabet as frozen sets for phonology and tokenization work:

from kreyolib import (
    ORAL_VOWELS,
    NASAL_VOWELS,
    CONSONANTS,
    SEMI_VOWELS,
    ALPHABET,
)

ORAL_VOWELS  # frozenset({'a', 'e', 'è', 'i', 'o', 'ò', 'ou'})
NASAL_VOWELS  # frozenset({'an', 'en', 'on', 'oun', 'in'})
CONSONANTS  # frozenset({'b', 'ch', 'd', 'f', 'g', 'h', 'j', 'k', 'l', 'm', 'n', 'ng', 'p', 'r', 's', 't', 'v', 'z'})
SEMI_VOWELS  # frozenset({'w', 'y', 'ui'})
ALPHABET  # Combines all

Normalization

Orthography (API)

Standardizes chat slang, archaic spellings, clitics, and article usage into modern IPN orthography.

from kreyolib.normalize.orthography import standardize_text

# -- Chat slang & abbreviations --
standardize_text("Bjr! Mw tap tann ou sou ban an, svp cheri ou knn c fèt mwen jodi a.")
# Bonjou! Mwen tap tann ou sou ban an, silvouplè cheri ou konn se fèt mwen jodi a.

# -- Article correction --
standardize_text("Mwen chita sou ban a. Mwen ap manje bannann la ki te sou tab lan.")
# Mwen chita sou ban an. Mwen ap manje bannann nan ki te sou tab la.

# -- Capitalization fixes --
standardize_text("mwen renmen bondye")
# Mwen renmen Bondye

With aggressive=True, older historical variations and non-standard spellings are folded in as well:

standardize_text("Nan lé monn mouin té pèdu nan péché; Min Jézu té sové-m.", aggressive=True)
# Nan le mond mwen te pèdi nan peche; Men Jezi te sovem.

Contractions (API)

Expands colloquial clitics (m'ap, y'ap, n') into formal standalone words.

from kreyolib.normalize.contractions import expand_contractions

expand_contractions("M'ap ale lakay nou paske yap tann nou pou n' al travay.")
# Mwen ap ale lakay nou paske yo ap tann nou pou nou al travay.

Diacritics (API)

Removes accent marks — useful for search indexes or legacy systems that expect plain ASCII.

from kreyolib.normalize.diacritics import strip_diacritics

strip_diacritics("Abèy yo ap vole sou òganizasyon an lè yo ale nan fèt la.")
# Abey yo ap vole sou oganizasyon an le yo ale nan fet la.

Tokenization

Sentence Splitter (API)

Splits text into sentences, respecting abbreviations, quotes, and parenthesized boundaries. Powered by yasbd-lib, with Haitian Creole rules extended with French abbreviations and sentence starters.

from kreyolib.tokenize.sentence import sent_tokenize

sent_tokenize("Alo mond. Koman ou ye? Mwen byen.")
# ['Alo mond.', 'Koman ou ye?', 'Mwen byen.']

sent_tokenize(
    'M. Dupont est un professeur. Li travay nan lekòl la. Li di: "Mwen pral vini demen." Apre sa, li ale.'
)
# ['M. Dupont est un professeur.', 'Li travay nan lekòl la.', 'Li di: "Mwen pral vini demen."', 'Apre sa, li ale.']

Word Tokenizer (API)

Splits text into word-level tokens while preserving abbreviations, hashtags, mentions, and URLs.

from kreyolib.tokenize.word import word_tokenize

word_tokenize("Dr. Jean-Louis t'ap travay U.S.A nan Yahoo!")
# ['Dr.', 'Jean-Louis', 't', "'", 'ap', 'travay', 'U.S.A', 'nan', 'Yahoo!']

word_tokenize("@Jhon Sak genla? ##myboy")
# ['@Jhon', 'Sak', 'genla', '?', '##myboy']

word_tokenize("www.google.com avèk Jhon@gmail.com.")
# ['www.google.com', 'avèk', 'Jhon', '@gmail.com', '.']

BPE Tokenizer (API)

Returns subword tokens, their IDs, and matching subword embeddings, which is useful for subword-level NLP tasks such as language modeling or sequence tagging on Haitian Creole text. Built on the BPEmb Haitian Creole model (5k-piece vocab, 100 dim vectors). See the BPE Source Code for details.

Requires the optional bpe extra (pip install kreyolib[bpe]), which brings in numpy and sentencepiece. The SentencePiece model and its 100-dimensional Word2Vec vectors ship with the package, so no download happens at runtime.

from kreyolib.tokenize.bpe import bpe_tokenize

result = bpe_tokenize("Mwen ap travay nan jaden an.")

result["tokens"]
# ['▁mwen', '▁ap', '▁travay', '▁nan', '▁jaden', '▁an', '.']

result["ids"]
# [1194, 438, 976, 31, 3863, 44, 4962]

result["embeddings"].shape
# (7, 100)

Set lowercase=False to preserve the original casing during tokenization.

bpe_tokenize("Mwen", lowercase=False)["tokens"]
# ['▁', 'M', 'wen']

Conversion

Number to Text (API)

Converts an integer or decimal into its Kreyòl word form. It also supports negative numbers and ordinal numbers.

from kreyolib.convert.num_to_text import num_to_text

print(num_to_text(223))  # 'de san venntwa'
print(num_to_text(1_000_000))  # 'yon milyon'
print(num_to_text(12.4))  # 'douz pwen kat'
print(num_to_text(-5))  # 'mwens senk'
print(num_to_text(400_034))  # 'kat san mil trannkat'
print(num_to_text(0.17))  # 'zewo pwen disèt'
print(num_to_text(0.014))  # 'zewo pwen zewo katòz'
print(num_to_text(42, ordinal=True))  # 'san vennkatryèm'
print(num_to_text(124, ordinal=True))  # 'karanndezyèm'

Text to Number (API)

Converts Kreyòl number words back into an integer or float. The converter supports negative numbers, decimals, and larger scales (mil, milyon...). By default, minor spelling variations are tolerated through fuzzy matching; pass fuzzy=False to require exact words only.

from kreyolib.convert.text_to_num import text_to_num

print(text_to_num("de san venntwa"))  # 223
print(text_to_num("yon milyon san uit"))  # 1000108
print(text_to_num("douz pwen kat"))  # 12.4
print(text_to_num("mwens de san"))  # -200
print(text_to_num("de mil de san"))  # 200200
print(text_to_num("de san de mil"))  # 202000
print(text_to_num("zewo pwen zewo uit"))  # 0.08
print(text_to_num("de san", fuzzy=False))  # 200 (exact words only)

try:
    text_to_num("de sann", fuzzy=False)  # "sann" is not an exact word
except ValueError:
    print("rejected (fuzzy matching disabled)")

Datetime to Text (API)

Converts a datetime or timedelta into Kreyòl date, time, or relative-time text.

from datetime import datetime, timedelta

from kreyolib.convert.datetime_to_text import datetime_to_text

print(datetime_to_text(datetime(2026, 9, 4)))
# 'vandredi 4 septanm 2026'

print(datetime_to_text(datetime(2023, 12, 3, 15, 30, 42)))
# 'dimanch 3 desanm 2023, 15:30:42'

print(datetime_to_text(-timedelta(weeks=4, days=8), relative=True))
# 'sa gen 1 mwa e 5 jou'

print(datetime_to_text(timedelta(weeks=12, days=3, hours=60), relative=True))
# 'nan 2 mwa, 4 semèn e 12 èdtan'

Relative conversion can express a duration from the current time.

print(datetime_to_text(timedelta(hours=5), relative=True))
# 'jodi a, nan 5 èdtan'

Text to Datetime (API)

Parses Kreyòl date and time expressions into a datetime object. It supports numeric timestamps, absolute dates, relative dates, relative durations, weekdays, periods, and common time expressions.

from kreyolib.convert.text_to_datetime import text_to_datetime

print(text_to_datetime("2026-01-08 22:33"))
# datetime(2026, 1, 8, 22, 33)

print(text_to_datetime("samdi 1 janvye 2019"))
# datetime(2019, 1, 1)

print(text_to_datetime("demen"))
# datetime(2026, 1, 2)

print(text_to_datetime("semèn pwochèn a 10h"))
# datetime(2026, 1, 8, 10, 0)

print(text_to_datetime("demen a dizè"))
# datetime(2026, 1, 2, 10, 0)

Relative expressions can describe durations, previous or upcoming periods, and weekdays.

print(text_to_datetime("sa gen yon ane"))
# datetime(2025, 1, 1)

print(text_to_datetime("sa gen 5 jou, kat semèn"))
# datetime(2025, 11, 29)

print(text_to_datetime("semèn pase"))
# datetime(2025, 12, 25)

print(text_to_datetime("madi pase"))
# datetime(2025, 12, 31)

print(text_to_datetime("mwa kap vini a"))
# datetime(2026, 2, 1)

Time expressions can be combined with relative or absolute date expressions.

print(text_to_datetime("avan yè a 10:45"))
# datetime(2025, 12, 30, 10, 45)

print(text_to_datetime("apre demen a 15è eka"))
# datetime(2026, 1, 3, 15, 15)

print(text_to_datetime("jedi pase a 3è edmi"))
# datetime(2025, 12, 26, 3, 30)

print(text_to_datetime("jedi pwochèn"))
# datetime(2026, 1, 8)

Advanced Models & Intelligence

POS Tagger (API)

Built on Universal Dependencies treebanks (Autogramm, Adolphe) with custom preprocessing and French-based proper noun handling. See the Tagger Source Code for training details.

Tag a raw sentence:

from kreyolib.tagger.pos import tag

tag("Map vini demen nan maten pou n al travay ansanm.")
# [('M', 'PRON'), ('ap', 'AUX'), ('vini', 'VERB'), ('demen', 'NOUN'), ('nan', 'ADP'),
#  ('maten', 'NOUN'), ('pou', 'ADP'), ('n', 'PRON'), ('al', 'VERB'), ('travay', 'VERB'),
#  ('ansanm', 'ADV'), ('.', 'PUNCT')]

Or a pre-tokenized list:

tag(["Mwen", "rele", "Jan", ",", "e", "mwen", "abite", "Okay", "."])
# [('Mwen', 'PRON'), ('rele', 'VERB'), ('Jan', 'PROPN'), (',', 'PUNCT'), ('e', 'CCONJ'),
#  ('mwen', 'PRON'), ('abite', 'VERB'), ('Okay', 'NOUN'), ('.', 'PUNCT')]

Corpus & Data

Language data bundled with kreyolib, importable from kreyolib.corpus.

from kreyolib.corpus import STOP_WORDS, CHAT_ABBRVS_MAP, SAMPLE_SENTENCES

Stop Words (API)

115 Haitian Creole stop words as a frozenset, derived from spaCy's Haitian Creole list under the MIT License. Useful as a default filter when you want function words out of the way.

len(STOP_WORDS)
# 115

"te" in STOP_WORDS
# True

Chat Abbreviations (API)

80 informal and SMS-style abbreviations mapped to their expanded forms, as a dict. Compiled from personal WhatsApp logs, so coverage is uneven and skewed toward casual registers. normalize_orthography applies this map for you.

CHAT_ABBRVS_MAP["avk"]
# 'avèk'

CHAT_ABBRVS_MAP["bjr"]
# 'bonjou'

Sample Sentences (API)

1,000 Haitian Creole sentences as a set, for tests, examples, and experimentation. Sampled from the CMU Haitian newswire corpus with a fixed seed, so the sample is reproducible.

len(SAMPLE_SENTENCES)
# 1000

sorted(SAMPLE_SENTENCES)[43]
# '400 konvansyonè yo ki tyeke nan otèl Cherry Hill Hilton nan se kèk nan pi gwo kliyan otèl lan.'

The sentences are a modified derivative of the CMU corpus, so the full CMU data license notice is embedded in the module source, as its condition 1 requires. Regenerate with python scripts/build_sent_corpus.py.

Roadmap & Progress / Plan Travay

• 1. Normalization & Preprocessing
  • Text standardization and modernization
  • Contraction expansion
  • Diacritics remover
• 2. Conversion
  • Number-to-text conversion in Kreyòl with bidirectional support
  • Datetime-to-text conversion in Kreyòl with bidirectional support
• 3. Corpus & Datasets
  • Stop words
  • Chat/informal abbreviations
  • Sentences
• 4. Advanced Models & Intelligence
  • Part-of-Speech (POS) tagging engine (ml)
  • Named Entity Recognition for Haitian entities (ml)
  • Lexicon-based sentiment analysis engine
  • Sentence/Next-word predictor
• 5. Tokenization & Segmentation
  • Context-aware Word tokenizer
  • Sentence boundary splitter (with support for mention, and hashtag)
  • Subword tokenization via Byte-Pair Encoding and rules
• 6. Phonetics & Syntax
  • Text-to-phonetics and IPA generation
  • CV syllabification engine
• 7. Spelling & Error Correction
  • Spell-checking engine

How People Can Contribute

For ways to contribute, see Contributing Guide.

This is a community-driven project to give Haitian Creole a stronger place in the AI and NLP ecosystem.