leptris (Python) — lxml-shaped bindings for libleptris

leptris wraps the libleptris C API (XML 1.0 parsing, XPath 1.0) using cffi in ABI mode, with a required C accelerator for Element allocation and the hot accessors (tag, text, attrib, get, indexing, sibling navigation and plain-path XPath evaluation). Wheels ship it compiled; sdist builds require a C compiler.

The pinned libleptris version lives in libleptris-version.txt (lockstep releases); CI builds it from the release tarball. Platform wheels bundle the compiled engine — pip install leptris works with no extra setup on macOS (x86_64/arm64), Linux (manylinux + musllinux, x86_64/aarch64) and Windows (amd64/arm64).

Requirements

• Python 3.9+
• cffi (installed automatically)
• nothing else for wheel installs — the pinned libleptris is vendored into leptris/_vendor/ by the wheel build (scripts/vendor_libleptris.sh). Wheels carry the compiled library AND its exact source (_vendor/engine/, with rebuild instructions in _vendor/README.md)
• source installs (pip install --no-binary leptris) need a C toolchain + cmake; the sdist bundles the pinned engine source and compiles it at install time — no LEPTRIS_LIB_PATH

Supported runtimes

• CPython 3.9+ (abi3 wheels; one wheel per platform covers every supported CPython, including future 3.x releases)
• Free-threaded CPython (3.13t/3.14t): supported — cp314t wheels ship for every platform, built with Py_GIL_DISABLED (version- specific; the accelerator's Fns table is write-once at import). Contract: one document per thread (documents may migrate between threads between calls); the shared SAX recorder is internally serialized
• PyPy / GraalPy: unsupported — the accelerator is required (single-mode by design); a pure-cffi fallback would contradict the binding's purpose. Revisit only on real demand.

Wheel platforms

Every release ships a universal wheel per platform (abi3, covering all CPython 3.9+) plus the matching free-threaded cp314t wheel and a source distribution. Each wheel runs the full test suite on its platform as part of the release build.

The pending rows are wired in the release pipeline and activate automatically when the engine releases land (leptris/leptris#1173, #1194).

For a development checkout (or to use your own libleptris build), the loader also accepts a shared library (libleptris.dylib / .so / .dll) on the loader path, or one named by LEPTRIS_LIB_PATH (which must name the library file — the loader dlopens it verbatim; it takes precedence over the vendored copy):

cmake -B build -S /path/to/leptris -DLEPTRIS_BUILD_SHARED=ON
cmake --build build --target leptris_shared
export LEPTRIS_LIB_PATH=/path/to/leptris/build/src/libleptris.dylib

Quick start

from leptris import fromstring, tostring

root = fromstring("<library><book id='1' lang='en'>Ulysses</book></library>")

root.tag                                # "library"
root[0].get("id")                       # "1"
root[0].attrib                          # {"id": "1", "lang": "en"}
root[0].text                            # "Ulysses"

root.xpath("count(//book)")             # 1.0
[b.text for b in root.findall("book")]  # ["Ulysses"]

tostring(root[0], encoding="unicode")   # "<book id=\"1\" lang=\"en\">Ulysses</book>"

Documents own the tree; use the context manager or close():

from leptris import parse

with parse("catalog.xml") as doc:
    doc.xpath("//book[@lang='en']")

Namespaces, variables, canonical XML and streaming:

root.xpath("//x:item", namespaces={"x": "urn:ex"})
root.xpath("//book[@id=$id]", variables={"id": "2"})
c14n(root, exclusive=True)

from leptris import sax
sax.parse(xml, handler)                       # one-shot
with sax.StreamingParser(handler) as parser:  # push, constant memory
    parser.feed(chunk, final=last)

XSLT and XPath version support

leptris.XSLT(stylesheet) compiles once, applies to any Document, and returns a Document; leptris.XPath(expression) compiles an XPath for repeated evaluation. Which language constructs work is decided by the engine — the matrix below is measured against libleptris 1.9.32 (audited through this binding; the upstream gap ledger is leptris/leptris#685).

XQuery 1.0 core is a first-class API since libleptris 1.9.64:

from leptris import Document, XQuery

with Document.parse("<r><item v='1'>alpha</item><item v='5'>beta</item></r>") as doc:
    XQuery("for $i in //item where $i/@v > 1 return string($i)")(doc)  # ['beta']
    XQuery("declare variable $n := 3; <out>{$n * 2}</out>")(doc)       # '<out>6</out>'
    XQuery("declare function local:dbl($x) { $x * 2 }; local:dbl(4)")(doc)  # 8.0

Schema-descriptor materialization (Plan, libleptris 1.9.162+)

Compile a descriptor once, materialize whole documents in one native pass — the tree-shaped ABI lutaml-model hosts consume (leptris/leptris#1039):

from leptris import Document, Plan

catalog = Plan({
    "element_name": "catalog",
    "children": [
        {"name": "item", "kind": "collection", "plan": {
            "element_name": "item",
            "attributes": {"id": {}, "price": {}},
            "children": [{"name": "title"}],
        }},
        {"name": "note", "kind": "raw"},
    ],
})
with Document.parse(xml) as doc:
    data = catalog(doc)   # {"attributes": {...}, "children": {...}}

Row kinds: scalar, collection (list), nested (element plan), raw (serialized subtree), content (mixed-content text runs, name may be ""), callback (PlanCallback(value, position, type_tag) — the escape hatch for host-side procs). Results are shape-stable (absent rows yield None / []); namespace forms via ns={"form": "none"|"exact"|"any", "uri": ...}, flags via {"mixed_content", "ordered", "cdata", "ns_lenient"}.

The default surface is the full XPath 3.1 grammar. To pin the strict XPath 1.0 surface (3.x-only syntax raises XPathError), pass version="1.0" — on Document.xpath, Element.xpath, and the compiled XPath class (XPath(expr, version="1.0")):

root.xpath("count(//book)", version="1.0")          # works — 1.0 grammar
root.xpath("//book ! @id", version="1.0")           # XPathError: 3.x syntax
XPath("//book[1]/@id", version="1.0")(root)         # compiled + strict

XPath 3.1 composition and XSLT 3.0 instructions flow through the existing API with zero binding change:

from leptris import Document, XPath, XSLT, tostring

with Document.parse("<r><item v='1'>alpha</item><item v='5'>beta</item></r>") as doc:
    doc.getroot().xpath("let $n := //item[2] return $n/@v")      # ['5']
    doc.getroot().xpath("(//item ! string(.)) => count()")        # 2.0

    style = XSLT("""<xsl:stylesheet version="3.0"
        xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
      <xsl:mode use-accumulators="depth"/>
      <xsl:accumulator name="depth" initial-value="0">
        <xsl:accumulator-rule match="*" phase="start" select="$value + 1"/>
        <xsl:accumulator-rule match="*" phase="end" select="$value - 1"/>
      </xsl:accumulator>
      <xsl:template match="/"><o>
        <xsl:iterate select="//item">
          <i d="{accumulator-before('depth')}"/>
        </xsl:iterate>
      </o></xsl:template>
    </xsl:stylesheet>""")
    print(tostring(style(doc), encoding="unicode"))
    # <o><i d="2"/><i d="2"/></o>   # items sit at depth 2 (r → item)

Unsupported constructs fail at XSLT() compile time or at evaluation with LeptrisError — except two instructions that still produce empty output instead of an error (xsl:map/xsl:map-entry, and xsl:value-of/@separator is ignored); tracked in the #685/#690 ledgers above.

HTML parsing

leptris.html (libleptris 1.9.75+) — tolerant HTML in the shape of lxml's etree.HTMLParser: implied end tags, void elements, case-insensitive names, unquoted attributes, the named-entity table, and a synthesized <html>/<head>/<body> wrapper:

from leptris import html

r = html.fromstring("<p>hello <b>world")   # <html><head/><body><p>hello <b>world</b></p></body></html>
with html.document("<td>c") as doc:
    doc.xpath("count(//td)")               # 1.0

Since libleptris 1.9.76 the output is byte-exact with lxml's etree.HTMLParser (minimized attributes are empty strings; no empty <head/> is emitted) — leptris/leptris#813. The one deliberate html4 divergence: a leading script/style run stays in body (the fragment shape), where lxml's libxml2 lifts it to head.

Two modes (libleptris 1.9.104+): the default html4 keeps this compatibility shape; mode="whatwg" selects the WHATWG-conformant engine — with libleptris 1.9.196 the WHATWG conformance corpus is at zero divergences (leptris/leptris#659 complete): tree construction, foster parenting, adoption agency, in-select/table scope, and the foreign-content rules all match the specification. leptris fully provides HTML.

Migrating from lxml

Layout

• leptris/_ffi.py — cdef mirror of the public headers + loader (the only place libleptris is declared)
• leptris/_engine.py — the compile-once lifecycle base shared by the compiled language objects (XSLT, XQuery, future RelaxNG)
• leptris/_results.py — the result model: engine results to Python values (nodeset wrapping, scalar conversion), shared by every evaluation surface
• leptris/_leptrisaccel.c — the C accelerator (abi3): allocates Elements and runs the hot accessors, subtree iteration, the parse and serialization seams, and the per-document element registry; bound to libleptris by the positional protocol in element.py
• leptris/element.py, document.py, node.py, xpath.py — the Python surface: queries, walks, documents; node.py exposes the full DOM (comments, CDATA, PIs) beneath the ElementTree shape
• leptris/api.py — fromstring/parse/tostring/c14n/ iterparse
• leptris/sax.py — SAX one-shot and streaming
• tests/ — pytest suite (pytest with LEPTRIS_LIB_PATH set)
• benchmarks/ — matrix vs lxml/ElementTree/minidom (pip install .[bench], then python -m benchmarks.matrix)

Memory model

The Document owns the whole tree and its pool. Accessor strings are copied into Python str at the boundary, so nothing depends on document lifetime after a call returns. Elements keep a reference to their Document, so the pool cannot be freed while any wrapper is alive. Prefer explicit close() / the context manager; __del__ is a refcounting safety net, not a contract. Using an element after its document is closed raises LeptrisError.

Versioning

libleptris-version.txt pins the library release the binding is built and tested against. Since 1.9.76.0 the package version is {c-full-semver}.{patch} — the pinned libleptris version plus a binding-local patch counter: lib 1.9.76 → 1.9.76.0, then 1.9.76.1 for binding-only fixes against the same pin, resetting the patch whenever the pin moves. (1.2.0–1.27.1 were the interim own-semver line.) pyproject.toml and leptris/__init__.py must agree at release time.

Publishing

Releases publish to PyPI via .github/workflows/release.yml, using PyPI trusted publishing (no stored credentials). The workflow runs on manual dispatch (ships the version in pyproject.toml) and is called by the libleptris release flow (publish: true), so every libleptris release ships the wheel.

Local development

python3 -m venv .venv
./.venv/bin/pip install --upgrade build pytest cffi
./.venv/bin/pip install -e .[test,bench]
LEPTRIS_LIB_PATH=/path/to/libleptris.dylib ./.venv/bin/python -m pytest tests/ -q