PENelopy

Pre-editorial normalization toolkit for Latin and Old French: a model-agnostic encapsulator around seq2seq normalizer models (currently comma-project/normalization-byt5-small), plus a character-level aligner that maps normalized output back onto the original text to produce TEI standoff XML and W3C Open Annotation JSON.

Install

uv add penelopy
# or, for the ONNX/quantized backend:
uv add "penelopy[onnx]"

CLI

penelopy normalize input.txt --model comma-project/normalization-byt5-small
penelopy quantize --model comma-project/normalization-byt5-small --output .cache/onnx --quantize
penelopy evaluate predictions.txt references.txt
penelopy align "tãt.⁊si" "tant et si"

# Or normalize via any OpenAI-compatible chat server (e.g. LM Studio):
penelopy normalize input.txt --backend lmstudio --model google/gemma-4-12b \
  --base-url http://localhost:1234/v1 --no-think

Library

from penelopy.models import load_model
from penelopy.pipeline import normalize_document

model = load_model("comma-project/normalization-byt5-small")
result = normalize_document("tãt.⁊si lamoit", model)
print(result.tei)
print(result.annotations)

See the full developer documentation, including the module architecture, at https://ponteineptique.github.io/PENelopy/.

License

Mozilla Public License 2.0.