trex for Python

The trex engine as a Python module: a regex-shaped pattern language whose alphabet is typed tokens (numbers, words, quoted strings, IPs, URLs, emails, timestamps, balanced bracket groups, and the rest), with long-distance binding, typed value predicates, and no catastrophic backtracking.

$ pip install trex-re

>>> import trex
>>> p = trex.Pattern(r"<\W:t>.*</=t>")
>>> m = p.find("say <div>hi</div> now")
>>> (m.start, m.end, m.text, m["t"])
(4, 17, '<div>hi</div>', 'div')
>>> [m.text for m in trex.Pattern(r"\N{>=100}").find_iter("5 50 500 5000")]
['500', '5000']
>>> trex.Pattern(r"\E:e").rewrite("[${e:domain}]", "mail bob@x.com now")
'mail [x.com] now'
>>> trex.Pattern(r"\E:e").rewrite(lambda m: m["e:user"].upper(), "mail bob@x.com now")
'mail BOB now'

Inputs and offsets

Every scan takes a str or bytes. A str is scanned as its UTF-8 bytes and reported in characters, so s[m.start:m.end] == m.text; bytes are scanned and reported as bytes. A match also carries byte_start and byte_end, its UTF-8 byte offsets, whatever the input was. Results come back in the input's type: str for str, bytes for bytes.

API

Scans release the interpreter lock while they run. The pattern language is documented in the pattern syntax reference.

Building

The wheel is built with maturin from this directory of the trex repository, with the gpu feature and the stable ABI from Python 3.11:

$ pip install maturin
$ maturin build --release -m python/Cargo.toml

It builds and passes its suite on Windows (win_amd64), Linux (manylinux_2_34_x86_64) and FreeBSD 15.0 (freebsd_15_0_release_amd64). There is no macOS build. A host without a CUDA driver runs the CPU engine; the device is probed once and its absence is not an error.