cliff-python 0.2.0

cliff-python

Python parser, serializer, validator and converter for CLIFF 1.0 / 1.1 — the Contextual Localization Integrated File Format.

CLIFF keeps translation context — namespace, clan, group path, entry id, type, emotion, width — next to every string, in a line-oriented format that stays cheap for translators and LLMs alike. cliff-python is the reference implementation of that format for Python: one typed data model, one parser, and every converter a localization pipeline needs.

The parser, serializer, validator and converters target CLIFF 1.1 and are checked against the normative examples of both 1.0 and 1.1 plus the cliff-test conformance fixtures.

Installation

pip install cliff-python

Optional converter support (YAML) is a separate extra, so the core package has no dependencies:

pip install "cliff-python[converters]"

Requires Python 3.11 or newer.

Quick start

import cliff_format

text = '''CLIFF 1.1
namespace: demo
clan: settings
source-language: en-US
target-language: zh-CN

[Video]
type: label

<Resolution>
source: "Resolution"
target: "分辨率"
status: final
'''

doc = cliff_format.parse(text)
print(doc.spec_version)              # 1.1 — the version the document declared
print(doc.header.namespace)          # demo
print(doc.groups[0].path)            # Video
print(doc.groups[0].entries[0].id)   # Resolution

print(cliff_format.serialize(doc))   # canonical CLIFF

for issue in cliff_format.validate(text):
    print(issue.line, issue.category, issue.message)

Strict and tolerant parsing

CLIFF 1.1 defines two readings of the same syntax, and cliff_format exposes both. Strict is the default; the mode is never chosen for you.

Strict (parse, load) rejects everything the grammar rejects, and accepts everything it accepts:

• identifiers may use A-Z a-z 0-9 _ - and never ., are case-sensitive, and are never rewritten — bad_id and BadVersion stay as written, because an identifier is a translation match key;
• tags (type, emotion, status, variant) are the one thing 1.1 did not relax: lowercase words from a closed vocabulary;
• one optional trailing , or ; per line is standard syntax, means nothing, and is never re-emitted by the serializer;
• a file-layout mismatch is a warning — the layout is a recommendation;
• resource limits are enforced.

Tolerant (parse(..., tolerant=True), parse_tolerant) adds the seven relaxations of Appendix C and reports every repair:

C.2.7 removes the quotes and nothing else: the enclosed text must be a name, no escape is processed, and the key is then checked against the legal keys of its scope exactly as a bare key is. A quoted word that is not a legal key, or a key quoted in a scope that does not allow it, is still an error (§C.5) — the relaxation cannot legalize a word.

C.2.5 applies to an identifier that begins with a name character (ignoring surrounding whitespace), or that uses the quoted form of C.2.4. Angle brackets that enclose anything else are not an entry marker: a stray closing tag </terms> — what a model writes when it treats a section like an XML element — or <.hidden> is rejected under C.5 rather than normalized. Without that limit the reading turned /terms into the entry terms and then failed the document on a required field of an entry the file never contained, which is the guessing C.5 forbids. An identifier that is empty or whitespace-only is unchanged: C.3 step 7 still gives the scope's fallback name.

Every repair is recorded, so nothing is fixed silently:

doc, corrections = cliff_format.parse_tolerant(text)
for c in corrections:
    print(c.line, c.category, c.message, c.before, "->", c.after)

Categories: list-shape, field-repeat, tag-quote, name-quote, name-normalized, id-collision, version, casing. A category names the operation, not the clause, so name-quote covers both a quoted entry id (C.2.4) and a quoted key (C.2.7).

The mode never guesses. Appendix C.5 lists what a tolerant parser MUST refuse — a missing required field, a word outside a closed vocabulary, an unbalanced ICU brace, a malformed line, an unsupported version — and those raise CliffParseError with a line number and a category, exactly as in strict mode.

Two guarantees make the mode usable in a pipeline:

# 1. the repaired document is valid under the strict grammar
once = cliff_format.serialize(doc)
assert not [i for i in cliff_format.validate(once) if i.category not in ("warning", "extension")]

# 2. canonical output is a fixed point
assert cliff_format.serialize(cliff_format.parse(once)) == once

Command line

cliff-format parse path/to/file.cliff
cliff-format serialize path/to/file.cliff
cliff-format validate path/to/file.cliff
cliff-format convert path/to/file.cliff --format po
cliff-format convert path/to/file.po --from po --format cliff

# Tolerant reading of a model's output, with the repair report on stderr
cliff-format parse path/to/model-output.cliff --tolerant
cliff-format validate path/to/model-output.cliff --tolerant
cliff-format convert path/to/model-output.cliff --tolerant --format json

# Opt-in checks: a layout mismatch as an error, style deviations as warnings
cliff-format validate path/to/file.cliff --check-layout
cliff-format validate path/to/file.cliff --style
cliff-format validate path/to/file.cliff --check-width

# Canonical output for a specific version, or with a canonical id style applied
cliff-format serialize path/to/file.cliff --spec-version 1.0
cliff-format serialize path/to/file.cliff --style

Supported convert formats: cliff, json, yaml, csv, po, xliff, fluent, android, ios.

validate exits 1 when the file has errors and 0 when it only has warnings, extension notes, corrections, or style findings. Every diagnostic carries a line number, a category, and the offending line.

Converters

Bidirectional for JSON, YAML, CSV, PO, XLIFF 2.x, Fluent, Android strings.xml and iOS Localizable.strings:

po_text = cliff_format.to_po(doc)
doc = cliff_format.from_po(po_text)

# XLIFF 2.0 / 2.1 / 2.2; 2.2 also writes the glossary module for variant: glossary
xml = cliff_format.to_xliff(doc, version="2.2")

CLIFF carries context as first-class data, and every converter moves that data through the target format's own documented metadata channel — never through an invented one:

Because formats without a group level cannot express inheritance, the exporters fold the group values into each entry (the effective context, type, emotion and max-width) so nothing is lost.

The reverse direction is deliberately conservative. An importer reads only the metadata the source format actually defines: namespaced cliff: entries become CLIFF attributes, an ordinary comment becomes context, and everything else falls back to a documented default (type: sentence, status derived from whether a translation exists). A converter never invents context, emotion, references, widths or reviewers that the source file did not contain.

Round-trip fidelity

JSON, YAML, CSV and XLIFF preserve the whole CLIFF data model, so cliff -> format -> cliff is lossless. The remaining formats have no slot for some CLIFF fields:

Importers keep foreign identifiers that are already valid CLIFF 1.1 names — InvSwordIron and bad_id are valid names, and rewriting one silently would break a translation memory. Only characters outside the name set are replaced, by the documented normalization of specification Appendix C.3, and duplicates are disambiguated with a numeric suffix so every generated document validates.

API

Diagnostics

Resource limits

parse and load accept max_lines, max_line_length and max_string_length; the defaults bound a document at 1 000 000 lines, 1 MiB per line and 1 MiB per decoded string. Exceeding a limit raises a located error instead of exhausting memory. The limits apply in strict and tolerant mode alike (§19).

Repository layout

cliff-python/
├── pyproject.toml
├── src/
│   └── cliff_format/
│       ├── identifiers.py   # name / tag / terminator rules (§5.5, §5.6, Appendix C.3)
│       ├── vocabulary.py    # the closed tag vocabularies (§12, §13)
│       ├── model.py         # dataclasses for the CLIFF data model
│       ├── parser.py        # single-pass line parser, strict and tolerant
│       ├── serializer.py    # canonical serializer
│       ├── validator.py     # semantic validation, inheritance helpers, style and layout checks
│       ├── converter.py     # bidirectional format converters
│       └── cli.py           # cliff-format command line
├── tools/
│   └── regenerate_examples.py
├── examples/
└── tests/

Conversion samples live in examples/cliff_to_<format>/ and examples/<format>_to_cliff/. The generated side is produced by tools/regenerate_examples.py; run it after changing a converter, and python tools/regenerate_examples.py --check verifies the samples are current.

Links

• CLIFF specification
• Conformance fixtures
• Changelog
• Issues

License

MIT — see LICENSE. The CLIFF wordmark under docs/assets/ comes from the CLIFF project and is licensed under the MIT License as well.