pyshacl-turbo

A Rust SHACL engine with Python bindings. Shape compilation, Core validation, SPARQL constraints, targets, rules and inference execute in Rust. There is no runtime dependency on pySHACL or owlrl. Native execution releases the Python GIL.

The goal is a complete drop-in replacement for pySHACL, with additional features and better performance. SHACL-JS is planned; current implementation work closes the non-JS gaps first. The pinned reference is pySHACL 0.40.1.

Turbo passes 403/403 non-JS cases (100%) in the locally corrected upstream suite: 400 unchanged behavioral cases, one corrected fixture, and 2 explicitly recorded native equivalents for Python implementation tests. The corrected fixture checks that invalid geometry data is rejected and that supplying its missing subclass relationship makes it valid. Both engines pass these assertions with equivalent reports; see the correction and ADR 0018. The pinned W3C suite passes 120/120 (100%) with document inputs and 119/120 with RDFLib Graph inputs. The latter retains a documented input-dependent pySHACL behavior. These are separate measurements; see compatibility and the case-level upstream report.

Beyond these suites, remote SPARQL endpoint inputs are unsupported, and complete inference output and exact report-text formatting still differ from pySHACL. See known limits for the remaining gaps and intentional differences.

The direct public-contract comparison measures the actual APIs and CLI without the upstream test bridge, including return values, RDF graph scope, report text, errors and mutations. It records mismatches and reference failures separately and publishes timings only when observations agree. The replacement contract also tracks imports, public classes, remote inputs, the HTTP server and future JS work.

Core and SPARQL patterns execute Python-compatible regexes natively, including lookarounds and backreferences. SPARQL REPLACE retains the pinned RDFLib replacement behavior; native callbacks and query results preserve exact literal terms. See ADR 0015 for the contract and remaining regex limits.

Count and length bounds retain exact arbitrary-precision values, with pySHACL's parameter-specific datatype rules. See ADR 0016 for integer and qualified-count behavior.

Install

python -m pip install pyshacl-turbo

Requires Python 3.11+. Wheels are available for Linux x86-64/ARM64 (glibc 2.28+), macOS Intel/Apple Silicon, and Windows x86-64. Rust is only needed when building from source. This is an experimental release; the compatibility measurements and remaining limits above still apply.

For optional PyOxigraph Store input:

python -m pip install 'pyshacl-turbo[oxigraph]'

Build

Requires Rust/Cargo, Python 3.11+ and uv. Development uses Python 3.13. The repository pins Rust 1.98.0 for compilation and formatting.

make dev
make check

Python

from pyshacl_turbo import compile_shapes, validate

conforms, report_graph, report_text = validate(
    "data.ttl", shacl_graph="shapes.ttl", advanced=True
)

# Compile once and reuse across independent datasets.
program = compile_shapes("shapes.ttl", advanced=True)
conforms, report_graph, report_text = program.validate("data.ttl")

Inputs include RDFLib Graph/Dataset objects, paths, URLs, file objects, RDF text and bytes. Supported formats include Turtle, N-Triples, N-Quads, TriG, RDF/XML, N3 and JSON-LD. Named graphs remain available to SPARQL GRAPH clauses; Core validation sees their union. Reports are RDFLib graphs by default.

PyOxigraph Store input is also supported. Install the optional oxigraph extra when PyOxigraph is not already installed:

from pyoxigraph import RdfFormat, Store
from pyshacl_turbo import validate

store = Store()
store.load(path="data.trig", format=RdfFormat.TRIG)
conforms, report, text = validate(store, shacl_graph="shapes.ttl", advanced=True)

The Store input path avoids an intermediate RDFLib data graph. Independent Python extensions exchange a native N-Quads snapshot, so it is not zero-copy. See Store input details. Named graphs and blank-node labels are preserved. Literal spellings already normalized by a caller's Store cannot be recovered. Store rule and inference additions commit atomically only with inplace=True, after successful execution. Only new quads are committed; concurrent deletion of input quads is not undone.

RDFS and OWL-RL inference are implemented natively and selected with inference="rdfs", "owlrl" or "both". Coverage is still being expanded; current inferred graphs do not always match the reference's complete closure. owlrl is installed only by the reference-test extra, not as a runtime backend.

Rust

The standalone pyshacl-turbo-engine crate accepts exact-term oxrdf Dataset values and oxigraph::Store inputs. CompiledShapes::validate_store and expand_store copy a consistent quad snapshot without RDF text conversion. Their inplace flag controls atomic addition of derived quads to the Store. Compiled programs are shareable across threads; parallelism is caller-controlled.

Shape compilation uses shacl-structural-model with Turbo's exact RDF terms, then prepares native queries, regexes and indexes for execution. The original graph remains available for reports and source identity. See ADR 0017.

Performance

The latest non-inference matrix measures the public-contract fixes in ADR 0020: Core validation, increasing shape counts, SHACL rules and SPARQL patterns, through both RDFLib and Store inputs. Five interleaved samples per configuration include boundary conversion and report construction; full RDF report parity is required before timing. It was run separately from this task's builds and tests; other workspace activity was observed and is recorded in the load note. Small API-call timings with stricter output/interface checks are in the direct comparison.

make benchmark

The compiler integration comparison records before/after compilation and validation timings, including reused shapes, with complete report parity required before timing.

The performance report compares RDFLib inputs and already loaded PyOxigraph Stores, including conversion and report construction. It includes shape reuse, increasing data and shape counts, SPARQL rules and reasoning. Every timed path first has to produce an equivalent complete report. Raw samples, versions and source hashes are in current.json. The original measurements are preserved in performance-baseline.json.

Measurements are workload-specific. Sparse targeting can be slower because Turbo must copy the input graph. Reasoning timings are provisional where the complete inferred graphs differ, even when validation reports agree. Reference backend errors are recorded without assigning a speedup. The older current.md run was made while other workspace builds were active; its raw samples show substantial scheduling variation. Sparse-target workloads remain slower with fresh RDFLib input. See the report's load notes before using its speedup ratios for capacity planning.

The last full matrix includes literal identity, report graph context and inference axiom fixes. It predates the recursion and undefined-function changes in ADR 0012; its recorded source hashes identify the measured checkpoint. Earlier matrices remain in benchmarks/. The newer Core measurements include the Python pattern compatibility changes in ADR 0013, at 100, 1,000 and 10,000 nodes. The SPARQL pattern measurements exercise native REGEX and REPLACE at 100 and 1,000 nodes through both RDFLib and Store inputs. The report records concurrent build load alongside the raw interleaved samples. These measurements precede ADR 0016's integer-parameter changes; their source hashes identify the measured implementation.

# Run alone, separately from builds and other tests.
.venv/bin/python benchmarks/check_core.py

# Full-report-gated SPARQL regex and replacement comparison.
.venv/bin/python benchmarks/matrix.py --kinds sparql-patterns --repeats 7 --output benchmarks/sparql-patterns.json

# Original SPARQL pipeline comparison, including both reference backends.
make benchmark-pipeline

Verification and architecture

Local regressions compare full reports, rule output, graph mutation, blank-node identity, namespace handling, Store inputs and errors. Pinned W3C and pySHACL assets live under tests/w3c and vendor/pyshacl respectively.

make coverage
make test-compatibility
make test-regex
make test-sparql-evaluator
make test-coverage
.venv/bin/python scripts/run_upstream.py
.venv/bin/python scripts/run_upstream.py --reference
.venv/bin/python scripts/audit_reference_fixture.py

The three workflow badges report required CI checks on main. Each workflow also runs on pull requests, daily, and on manual dispatch:

The Python coverage report measures the complete pyshacl_turbo Python package with pytest-cov, including uncovered files and branches. CI requires 100% line and branch coverage on Linux Python 3.13. XML, JSON and HTML reports are uploaded for each run; passing main runs publish the HTML report and measured percentage badge to GitHub Pages. make test-coverage generates the same report locally in htmlcov/. This measures Python API/CLI code coverage; it does not measure Rust code coverage or SHACL compatibility. No package files are omitted from the denominator.

A green Compatibility badge means the documented baseline holds: 403/403 locally corrected upstream passes, 120/120 W3C document passes and 119/120 W3C Graph passes. Every upstream case must pass; the one local fixture correction and two native equivalents remain identified in the report. The corrected fixture must also pass its independent two-engine audit. The W3C Graph divergence remains a strict expected failure. New failures and unexpected Graph passes fail CI. These badges do not claim complete pySHACL compatibility.

The same Compatibility job also runs the direct public-contract catalogue. Previously matching cases and fields must remain equal; recorded differences and reference failures remain visible in its JSON and Markdown artifacts.

Upstream allowances which turn failures into passes are disabled for both engines. The bridge blocks reference validation and rule execution. Original comparators are test infrastructure only. The private workload in the motivating screenshot has not been tested.

Architectural decisions are in docs/adr. The exact-term dataset avoids premature literal canonicalization. Documented local Rust dependency patches preserve RDFLib string identity and SHACL SPARQL prebinding.

PRs use conventional titles and squash merges; main requires signed commits, passing prepare, test, compatibility, regex, and Conventional Commit checks, and the configured owner review. Repository maintenance follows the public Legra release policy and the shared legra-ai/github-actions pipeline. Release artifacts and recovery are described in releasing.

License

Original project code is dual-licensed under MIT or Apache-2.0, at your option (MIT OR Apache-2.0). Copyright 2026 DataRoad Inc, Delaware, USA, trading as Legra. Vendored code and assets retain their licenses and provenance.