pyshacl-turbo

A Rust SHACL engine with Python bindings. Shape compilation, Core validation, SPARQL constraints, targets, rules and inference execute in Rust. There is no runtime dependency on pySHACL or owlrl. Native execution releases the Python GIL.

Compatibility work targets pySHACL 0.40.1, excluding SHACL-JS. It is incomplete. The locally corrected upstream suite passes 403/403 cases (100%): 400 unchanged behavioral cases, one corrected fixture, and 2 explicitly recorded native equivalents for Python implementation tests. The corrected fixture checks that invalid geometry data is rejected and that supplying its missing subclass relationship makes it valid. Both engines pass these assertions with equivalent reports; see the correction and ADR 0018. The pinned W3C suite passes 120/120 (100%) with document inputs and 119/120 with RDFLib Graph inputs. The latter retains a documented input-dependent pySHACL behavior. These are separate measurements; see compatibility and the case-level upstream report.

Core and SPARQL patterns execute Python-compatible regexes natively, including lookarounds and backreferences. SPARQL REPLACE retains the pinned RDFLib replacement behavior; native callbacks and query results preserve exact literal terms. See ADR 0015 for the contract and remaining regex limits.

Count and length bounds retain exact arbitrary-precision values, with pySHACL's parameter-specific datatype rules. See ADR 0016 for integer and qualified-count behavior.

Build

Requires Rust/Cargo, Python 3.11+ and uv. Development uses Python 3.13. The repository pins Rust 1.98.0 for compilation and formatting.

make dev
make check

Python

from pyshacl_turbo import compile_shapes, validate

conforms, report_graph, report_text = validate(
    "data.ttl", shacl_graph="shapes.ttl", advanced=True
)

# Compile once and reuse across independent datasets.
program = compile_shapes("shapes.ttl", advanced=True)
conforms, report_graph, report_text = program.validate("data.ttl")

Inputs include RDFLib Graph/Dataset objects, paths, URLs, file objects, RDF text and bytes. Supported formats include Turtle, N-Triples, N-Quads, TriG, RDF/XML, N3 and JSON-LD. Named graphs remain available to SPARQL GRAPH clauses; Core validation sees their union. Reports are RDFLib graphs by default.

PyOxigraph Store input is also supported. Install the optional oxigraph extra when PyOxigraph is not already installed:

from pyoxigraph import RdfFormat, Store
from pyshacl_turbo import validate

store = Store()
store.load(path="data.trig", format=RdfFormat.TRIG)
conforms, report, text = validate(store, shacl_graph="shapes.ttl", advanced=True)

The Store input path avoids an intermediate RDFLib data graph. Independent Python extensions exchange a native N-Quads snapshot, so it is not zero-copy. See Store input details. Named graphs and blank-node labels are preserved. Literal spellings already normalized by a caller's Store cannot be recovered. Store rule and inference additions commit atomically only with inplace=True, after successful execution. Only new quads are committed; concurrent deletion of input quads is not undone.

RDFS and OWL-RL inference are implemented natively and selected with inference="rdfs", "owlrl" or "both". Coverage is still being expanded; current inferred graphs do not always match the reference's complete closure. owlrl is installed only by the reference-test extra, not as a runtime backend.

Rust

The standalone pyshacl-turbo-engine crate accepts exact-term oxrdf Dataset values and oxigraph::Store inputs. CompiledShapes::validate_store and expand_store copy a consistent quad snapshot without RDF text conversion. Their inplace flag controls atomic addition of derived quads to the Store. Compiled programs are shareable across threads; parallelism is caller-controlled.

Shape compilation uses shacl-structural-model with Turbo's exact RDF terms, then prepares native queries, regexes and indexes for execution. The original graph remains available for reports and source identity. See ADR 0017.

Performance

make benchmark

The compiler integration comparison records before/after compilation and validation timings, including reused shapes, with complete report parity required before timing.

The performance report compares RDFLib inputs and already loaded PyOxigraph Stores, including conversion and report construction. It includes shape reuse, increasing data and shape counts, SPARQL rules and reasoning. Every timed path first has to produce an equivalent complete report. Raw samples, versions and source hashes are in current.json. The original measurements are preserved in performance-baseline.json.

Measurements are workload-specific. Sparse targeting can be slower because Turbo must copy the input graph. Reasoning timings are provisional where the complete inferred graphs differ, even when validation reports agree. Reference backend errors are recorded without assigning a speedup. The latest run was made while other workspace builds were active; its raw samples show substantial scheduling variation. Sparse-target workloads remain slower with fresh RDFLib input. See the report's load notes before using its speedup ratios for capacity planning.

The last full matrix includes literal identity, report graph context and inference axiom fixes. It predates the recursion and undefined-function changes in ADR 0012; its recorded source hashes identify the measured checkpoint. Earlier matrices remain in benchmarks/. The newer Core measurements include the Python pattern compatibility changes in ADR 0013, at 100, 1,000 and 10,000 nodes. The SPARQL pattern measurements exercise native REGEX and REPLACE at 100 and 1,000 nodes through both RDFLib and Store inputs. The report records concurrent build load alongside the raw interleaved samples. These measurements precede ADR 0016's integer-parameter changes; their source hashes identify the measured implementation.

# Run alone, separately from builds and other tests.
.venv/bin/python benchmarks/check_core.py

# Full-report-gated SPARQL regex and replacement comparison.
.venv/bin/python benchmarks/matrix.py --kinds sparql-patterns --repeats 7 --output benchmarks/sparql-patterns.json

# Original SPARQL pipeline comparison, including both reference backends.
make benchmark-pipeline

Verification and architecture

Local regressions compare full reports, rule output, graph mutation, blank-node identity, namespace handling, Store inputs and errors. Pinned W3C and pySHACL assets live under tests/w3c and vendor/pyshacl respectively.

make coverage
make test-compatibility
make test-regex
make test-sparql-evaluator
make test-coverage
.venv/bin/python scripts/run_upstream.py
.venv/bin/python scripts/run_upstream.py --reference
.venv/bin/python scripts/audit_reference_fixture.py

The three workflow badges report required CI checks on main. Each workflow also runs on pull requests, daily, and on manual dispatch:

The Python coverage report measures the complete pyshacl_turbo Python package with pytest-cov, including uncovered files and branches. CI requires 100% line and branch coverage on Linux Python 3.13. XML, JSON and HTML reports are uploaded for each run; passing main runs publish the HTML report and measured percentage badge to GitHub Pages. make test-coverage generates the same report locally in htmlcov/. This measures Python API/CLI code coverage; it does not measure Rust code coverage or SHACL compatibility. No package files are omitted from the denominator.

A green Compatibility badge means the documented baseline holds: 403/403 locally corrected upstream passes, 120/120 W3C document passes and 119/120 W3C Graph passes. Every upstream case must pass; the one local fixture correction and two native equivalents remain identified in the report. The corrected fixture must also pass its independent two-engine audit. The W3C Graph divergence remains a strict expected failure. New failures and unexpected Graph passes fail CI. These badges do not claim complete pySHACL compatibility.

Upstream allowances which turn failures into passes are disabled for both engines. The bridge blocks reference validation and rule execution. Original comparators are test infrastructure only. The private workload in the motivating screenshot has not been tested.

Architectural decisions are in docs/adr. The exact-term dataset avoids premature literal canonicalization. Documented local Rust dependency patches preserve RDFLib string identity and SHACL SPARQL prebinding.

PRs use conventional titles and squash merges; main requires signed commits, passing test, compatibility, regex, and Conventional Commit checks, and the configured owner review. Repository maintenance follows the public Legra release policy and the shared legra-ai/github-actions pipeline. Release artifacts and recovery are described in releasing.

License

Original project code is dual-licensed under MIT or Apache-2.0, at your option (MIT OR Apache-2.0). Copyright 2026 DataRoad Inc, Delaware, USA, trading as Legra. Vendored code and assets retain their licenses and provenance.