leptris (Python) — lxml-shaped bindings for libleptris

leptris wraps the libleptris C API (XML 1.0 parsing, XPath 1.0) using cffi in ABI mode, with a required C accelerator for Element allocation and the hot accessors (tag, text, attrib, get, indexing, sibling navigation and plain-path XPath evaluation). Wheels ship it compiled; sdist builds require a C compiler.

The pinned libleptris version lives in libleptris-version.txt (lockstep releases); CI builds it from the release tarball. Platform wheels bundle the compiled engine — pip install leptris works with no extra setup on macOS (x86_64/arm64), Linux (manylinux + musllinux, x86_64/aarch64) and Windows (amd64/arm64).

Requirements

• Python 3.9+
• cffi (installed automatically)
• nothing else for wheel installs — the pinned libleptris is vendored into leptris/_vendor/ by the wheel build (scripts/vendor_libleptris.sh). Wheels carry the compiled library AND its exact source (_vendor/engine/, with rebuild instructions in _vendor/README.md)
• source installs (pip install --no-binary leptris) need a C toolchain + cmake; the sdist bundles the pinned engine source and compiles it at install time — no LEPTRIS_LIB_PATH

Supported runtimes

• CPython 3.9+ (abi3 wheels; one wheel per platform covers every supported CPython, including future 3.x releases)
• Free-threaded CPython (3.13t/3.14t): supported — cp314t wheels ship for every platform, built with Py_GIL_DISABLED (version- specific; the accelerator's Fns table is write-once at import). Contract: one document per thread (documents may migrate between threads between calls); the shared SAX recorder is internally serialized
• PyPy / GraalPy: unsupported — the accelerator is required (single-mode by design); a pure-cffi fallback would contradict the binding's purpose. Revisit only on real demand.

Wheel platforms

Every release ships a universal wheel per platform (abi3, covering all CPython 3.9+) plus the matching free-threaded cp314t wheel and a source distribution. Each wheel runs the full test suite on its platform as part of the release build.

Every leg is required: a red leg fails the release before publish.

For a development checkout (or to use your own libleptris build), the loader also accepts a shared library (libleptris.dylib / .so / .dll) on the loader path, or one named by LEPTRIS_LIB_PATH (which must name the library file — the loader dlopens it verbatim; it takes precedence over the vendored copy):

cmake -B build -S /path/to/leptris -DLEPTRIS_BUILD_SHARED=ON
cmake --build build --target leptris_shared
export LEPTRIS_LIB_PATH=/path/to/leptris/build/src/libleptris.dylib

Quick start

from leptris import fromstring, tostring

root = fromstring("<library><book id='1' lang='en'>Ulysses</book></library>")

root.tag                                # "library"
root[0].get("id")                       # "1"
root[0].attrib                          # {"id": "1", "lang": "en"}
root[0].text                            # "Ulysses"

root.xpath("count(//book)")             # 1.0
[b.text for b in root.findall("book")]  # ["Ulysses"]

tostring(root[0], encoding="unicode")   # "<book id=\"1\" lang=\"en\">Ulysses</book>"

Documents own the tree; use the context manager or close():

from leptris import parse

with parse("catalog.xml") as doc:
    doc.xpath("//book[@lang='en']")

Namespaces, variables, canonical XML and streaming:

root.xpath("//x:item", namespaces={"x": "urn:ex"})
root.xpath("//book[@id=$id]", variables={"id": "2"})
c14n(root, exclusive=True)

from leptris import sax
sax.parse(xml, handler)                       # one-shot
with sax.StreamingParser(handler) as parser:  # push, constant memory
    parser.feed(chunk, final=last)

for rec in sax.records(xml):                  # one-crossing records tape
    ...                                       # (leptris 1.9.237.1+)

XSLT and XPath version support

leptris.XSLT(stylesheet) compiles once, applies to any Document, and returns a Document; leptris.XPath(expression) compiles an XPath for repeated evaluation. Which language constructs work is decided by the engine — the matrix below is measured against libleptris 1.9.32 (audited through this binding; the upstream gap ledger is leptris/leptris#685).

XQuery 1.0 core is a first-class API since libleptris 1.9.64:

from leptris import Document, XQuery

with Document.parse("<r><item v='1'>alpha</item><item v='5'>beta</item></r>") as doc:
    XQuery("for $i in //item where $i/@v > 1 return string($i)")(doc)  # ['beta']
    XQuery("declare variable $n := 3; <out>{$n * 2}</out>")(doc)       # [<Element 'out'>] — real nodes (1.9.314+); tostring() for markup
    XQuery("declare function local:dbl($x) { $x * 2 }; local:dbl(4)")(doc)  # 8.0

Schema-descriptor materialization (Plan, libleptris 1.9.162+)

Compile a descriptor once, materialize whole documents in one native pass — the tree-shaped ABI lutaml-model hosts consume (leptris/leptris#1039):

from leptris import Document, Plan

catalog = Plan({
    "element_name": "catalog",
    "children": [
        {"name": "item", "kind": "collection", "plan": {
            "element_name": "item",
            "attributes": {"id": {}, "price": {}},
            "children": [{"name": "title"}],
        }},
        {"name": "note", "kind": "raw"},
    ],
})
with Document.parse(xml) as doc:
    data = catalog(doc)   # {"attributes": {...}, "children": {...}}

Row kinds: scalar, collection (list), nested (element plan), raw (serialized subtree), content (mixed-content text runs, name may be ""), callback (PlanCallback(value, position, type_tag) — the escape hatch for host-side procs). Results are shape-stable (absent rows yield None / []); namespace forms via ns={"form": "none"|"exact"|"any", "uri": ...}, flags via {"mixed_content", "ordered", "cdata", "ns_lenient"}.

Serialize a walk (serialize) or a host-assembled flat op program (serialize_ops) back to XML; indent/indent_unit pretty-print element boundaries (1.9.341+, #425). document_ops(ops) and to_document(element_or_document) materialize the same results as native Document trees — attributes verbatim, root namespace declarations riding as xmlns attributes — with no string round trip (1.9.341+, #426):

doc = catalog.document_ops([
    ("element", 0, None), ("attr", "id", "c1"),
    ("element", 0, 0), ("attr", "id", "i1"),
    ("scalar", "first"), ("end",), ("end",),
])
doc.getroot()[0].tag   # "item"

The default surface is the full XPath 3.1 grammar. To pin the strict XPath 1.0 surface (3.x-only syntax raises XPathError), pass version="1.0" — on Document.xpath, Element.xpath, and the compiled XPath class (XPath(expr, version="1.0")):

root.xpath("count(//book)", version="1.0")          # works — 1.0 grammar
root.xpath("//book ! @id", version="1.0")           # XPathError: 3.x syntax
XPath("//book[1]/@id", version="1.0")(root)         # compiled + strict

XPath 3.1 composition and XSLT 3.0 instructions flow through the existing API with zero binding change:

from leptris import Document, XPath, XSLT, tostring

with Document.parse("<r><item v='1'>alpha</item><item v='5'>beta</item></r>") as doc:
    doc.getroot().xpath("let $n := //item[2] return $n/@v")      # ['5']
    doc.getroot().xpath("(//item ! string(.)) => count()")        # 2.0

    style = XSLT("""<xsl:stylesheet version="3.0"
        xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
      <xsl:mode use-accumulators="depth"/>
      <xsl:accumulator name="depth" initial-value="0">
        <xsl:accumulator-rule match="*" phase="start" select="$value + 1"/>
        <xsl:accumulator-rule match="*" phase="end" select="$value - 1"/>
      </xsl:accumulator>
      <xsl:template match="/"><o>
        <xsl:iterate select="//item">
          <i d="{accumulator-before('depth')}"/>
        </xsl:iterate>
      </o></xsl:template>
    </xsl:stylesheet>""")
    print(tostring(style(doc), encoding="unicode"))
    # <o><i d="2"/><i d="2"/></o>   # items sit at depth 2 (r → item)

Unsupported constructs fail at XSLT() compile time or at evaluation with LeptrisError — except two instructions that still produce empty output instead of an error (xsl:map/xsl:map-entry, and xsl:value-of/@separator is ignored); tracked in the #685/#690 ledgers above.

HTML parsing

leptris.html (libleptris 1.9.75+) — tolerant HTML in the shape of lxml's etree.HTMLParser: implied end tags, void elements, case-insensitive names, unquoted attributes, the named-entity table, and a synthesized <html>/<head>/<body> wrapper:

from leptris import html

r = html.fromstring("<p>hello <b>world")   # <html><head/><body><p>hello <b>world</b></p></body></html>
with html.document("<td>c") as doc:
    doc.xpath("count(//td)")               # 1.0

Since libleptris 1.9.76 the output is byte-exact with lxml's etree.HTMLParser (minimized attributes are empty strings; no empty <head/> is emitted) — leptris/leptris#813. The one deliberate html4 divergence: a leading script/style run stays in body (the fragment shape), where lxml's libxml2 lifts it to head.

Two modes (libleptris 1.9.104+): the default html4 keeps this compatibility shape; mode="whatwg" selects the WHATWG-conformant engine — with libleptris 1.9.196 the WHATWG conformance corpus is at zero divergences (leptris/leptris#659 complete): tree construction, foster parenting, adoption agency, in-select/table scope, and the foreign-content rules all match the specification. leptris fully provides HTML.

Building HTML works like building XML, on an HTML-mode document — serialization emits void elements without closing tags and keeps <br>-style names un-XHTML-ed (libleptris 1.9.225+):

from leptris import html
import io

doc = html.create()
p = doc.create_element("p", {"class": "intro"})
doc.set_root(p)
p.text = "hello"
buf = io.BytesIO(); doc.save_html(buf)   # <p class="intro">hello</p>

tostring(node, method="html") forces HTML output for any tree; HTML-parsed and html.create() documents serialize as HTML by default.

Migrating from lxml

Layout

• leptris/_ffi.py — cdef mirror of the public headers + loader (the only place libleptris is declared)
• leptris/_engine.py — the compile-once lifecycle base shared by the compiled language objects (XSLT, XQuery, future RelaxNG)
• leptris/_results.py — the result model: engine results to Python values (nodeset wrapping, scalar conversion), shared by every evaluation surface
• leptris/_leptrisaccel.c — the C accelerator (abi3): allocates Elements and runs the hot accessors, subtree iteration, the parse and serialization seams, and the per-document element registry; bound to libleptris by the positional protocol in element.py
• leptris/element.py, document.py, node.py, xpath.py — the Python surface: queries, walks, documents; node.py exposes the full DOM (comments, CDATA, PIs) beneath the ElementTree shape
• leptris/api.py — fromstring/parse/tostring/c14n/ iterparse
• leptris/sax.py — SAX one-shot and streaming
• tests/ — pytest suite (pytest with LEPTRIS_LIB_PATH set)
• benchmarks/ — matrix vs lxml/ElementTree/minidom (pip install .[bench], then python -m benchmarks.matrix)

Memory model

The Document owns the whole tree and its pool. Accessor strings are copied into Python str at the boundary, so nothing depends on document lifetime after a call returns. Elements keep a reference to their Document, so the pool cannot be freed while any wrapper is alive. Prefer explicit close() / the context manager; __del__ is a refcounting safety net, not a contract. Using an element after its document is closed raises LeptrisError.

Versioning

libleptris-version.txt pins the library release the binding is built and tested against. Since 1.9.76.0 the package version is {c-full-semver}.{patch} — the pinned libleptris version plus a binding-local patch counter: lib 1.9.76 → 1.9.76.0, then 1.9.76.1 for binding-only fixes against the same pin, resetting the patch whenever the pin moves. (1.2.0–1.27.1 were the interim own-semver line.) pyproject.toml and leptris/__init__.py must agree at release time.

Publishing

Releases publish to PyPI via .github/workflows/release.yml, using PyPI trusted publishing (no stored credentials). The workflow runs on manual dispatch (ships the version in pyproject.toml) and is called by the libleptris release flow (publish: true), so every libleptris release ships the wheel.

Local development

python3 -m venv .venv
./.venv/bin/pip install --upgrade build pytest cffi
./.venv/bin/pip install -e .[test,bench]
LEPTRIS_LIB_PATH=/path/to/libleptris.dylib ./.venv/bin/python -m pytest tests/ -q