Sparse Attention Fabric

Sparse Attention Fabric is an evidence-first PyTorch execution planner for exact causal-window attention. The initial 0.1.0 release is intentionally narrow: it chooses among causal SDPA, masked SDPA, and compiled FlexAttention while exposing its decision and preserving PyTorch autograd.

This release is an executable hypothesis, not a performance claim. No controlled GPU result is bundled yet. Numeric thresholds in the M0 protocol are pre-registered targets. saf-m0-validate --release checks a supplied matrix's structure and semantics but does not authenticate its origin; a result becomes publishable only after validation and independent GitHub-attestation verification against the exact controlled workflow, default-branch ref, and candidate commit.

What ships in 0.1.0

• An immutable CausalWindow pattern with exact token and block semantics.
• A small, inspectable, cached ExecutionFabric policy.
• Fail-closed CUDA execution through public PyTorch APIs.
• A locked 19-case, four-implementation benchmark matrix with 380 fresh-process rows.
• Raw-sample JSONL evidence, a JSON Schema, correctness checks, and release gates.
• A deterministic, explicitly non-release A100 feasibility model for hardware-free auditing.
• CPU contract tests plus a manually dispatched A100 evidence workflow.

M0 is not a general sparse tensor compiler, a custom CUDA kernel library, or an MoE runtime. See the architecture for the deliberate boundaries and the competitive landscape for the primary-source decision record. The evidence-gated roadmap states what can follow a positive, negative, or invalid controlled run.

Install

Planning and contract tests have no runtime dependency:

python -m pip install .

Evidence validation adds JSON Schema support:

python -m pip install '.[validation]'

CUDA execution requires the PyTorch 2.13 build appropriate for the machine. In controlled environments, install PyTorch from the official channel selected for the CUDA driver, then install this project without allowing pip to replace it:

python -m pip install --no-deps .
python -c 'import torch; print(torch.__version__, torch.version.cuda)'

The cuda and benchmark extras express compatible version constraints, but they cannot choose the correct CUDA wheel index for a particular host.

Use

from m0 import CausalWindow, ExecutionFabric

fabric = ExecutionFabric(max_cache_entries=32)
pattern = CausalWindow(window=512)
output = fabric(query, key, value, pattern)
plan = fabric.plan(
    pattern,
    sequence_length=query.shape[-2],
    dtype=str(query.dtype).removeprefix("torch."),
)
print(plan.backend, plan.rationale)

# Optional for a tight loop: validates once and retains metadata, not tensors.
prepared = fabric.prepare(query, key, value, pattern)
next_output = prepared(next_query, next_key, next_value)

Inputs must be contiguous CUDA BHLD tensors with FP16 or BF16 dtype, equal query/KV head counts and lengths, and head/value dimensions of 64 or 128. Unsupported inputs raise an error instead of silently changing semantics. Prepared calls reject a changed shape, stride, device, dtype, pattern, or autograd context; call prepare again when any execution metadata changes. The entry limit applies independently to plans, masks, compiled Flex functions, prepared executors, and device capabilities. Slow-path insertions evict the oldest entry; hot prepared-cache hits remain lock-free. Inspect the configured limit and current occupancy with fabric.cache_info().

For a complete projection-to-attention-to-output module, see the Transformer-style integration example.

Validate locally

python -m unittest discover -s tests -p 'test_*.py' -v
python -m json.tool m0/core_cases.json >/dev/null
python -m json.tool m0/result.schema.json >/dev/null
python -m json.tool m0/feasibility.schema.json >/dev/null
saf-m0 --help
saf-m0-model --help
saf-m0-validate --help

Run the full benchmark only on a controlled Linux CUDA host. The canonical protocol, operator attestations, commands, interpretation rules, and known limitations are in the M0 evidence guide and benchmark operations guide.

When the target GPU is unavailable, run saf-m0-model to audit exact pair counts, block rounding, visible storage, and peak-rate sensitivity envelopes. The modeled-validation guide states what the model can and cannot establish. Its output is structurally barred from release validation.

Evidence and claims

Every release-quality performance statement must link to the immutable JSONL, validated summary, verified GitHub attestations, exact commit, hardware/software environment, and timing scope. A detached artifact or checksum is not proof of a controlled run. Unsupported, OOM, correctness-failure, and negative-performance rows are part of the result and must not be removed. Modeled hardware results may guide engineering but may never be reported as measured CUDA performance. The direct baselines are SDPA, generic FlexAttention, and native PyTorch varlen_attn; the latter compiles its complete BHLD-to-THD-to-native-call-to-BHLD adapter and keeps all required layout work inside every timed sample. NATTEN is an external head-to-head prerequisite for any future ecosystem-wide or best-in-class claim, not a package dependency.

Contributing and security

See CONTRIBUTING.md for development and benchmark rules, SECURITY.md for private vulnerability reporting, and docs/releasing.md for the release process. The 0.1.0 assurance record documents the pre-release security and red-team disposition. Participation is governed by the Contributor Covenant.

Licensed under Apache-2.0. See LICENSE and NOTICE.