shapestate 0.1.0

Enrollment-based detection of substrate corruption (activation noise, weight quantization) in language models.

Shapestate

Shapestate detects substrate corruption in language models by reading their internal state. Activation noise and weight quantization shift a model's internal geometry while its output can still parse cleanly. Shapestate enrolls a per-model sensitivity profile from a model's own healthy and corrupt captures, then monitors new runs and flags the corruption type.

The central result of the study behind it: detection loci and thresholds differ across models and have to be enrolled per model. One method and a set of structural laws carry across models. The method held across seven models in four architecture families.

What the demo prints

The bundled thirty-second demo scores held-out runs of model F (Mistral-7B-Instruct-v0.3) against F's enrolled profile, on CPU, with no download:

Shapestate demo -- profile F (d=4096, weakest=int8)
scoring held-out runs [15, 16, 17, 18, 19] of task 'ref' (enrollment used runs 0-14)

  fp16   [healthy control] -> NO FLAGS
  noise  [corruption     ] -> flags noise(margin +74sd, norm-z 12), ...
  int4   [corruption     ] -> flags ..., int4(margin +117sd, norm-z 8), ...
  int8   [corruption     ] -> flags int8(margin +21sd, norm-z 10)

The healthy condition stays clean. Each corruption flags its own type with a large margin (noise +74sd, int4 +117sd, int8 +21sd). The own-type margin is the largest in every corrupt condition. Corruptions share direction, so a strong corruption can also raise a smaller cross-type margin.

Quickstart, tier 1: the thirty-second demo (CPU only)

No GPU, no model download, no torch.

pip install -r requirements-demo.txt
python demo.py

demo.py applies the same decision rule as monitor.py to pre-extracted band-mean and peak-layer-norm vectors, so it needs only numpy. The demo data derives only from model F, which is Apache-2.0 licensed.

You can also install the tool and get console scripts:

pip install .          # provides shapestate-enroll, shapestate-monitor, shapestate-demo
shapestate-demo        # the demo data ships with the install, so this runs from any directory

The capture path needs a CUDA GPU: pip install "shapestate[capture]" plus a torch build from https://pytorch.org .

Quickstart, tier 2: enroll your own model

This path needs a GPU (see Hardware Requirements). It captures the standard battery, builds a cache, enrolls a profile, then monitors. Capture is the heavy step.

1. Install the full dependencies:

   pip install -r requirements.txt
   

2. Capture the battery for your model. The stage scripts under reproduce/ are the reference recipe: four tasks, five conditions, thirty runs, greedy decoding, forty-eight new tokens, full hidden states. Copy the closest stage script (for example reproduce/bd_maxcap_b.py), set the model name, and run it to produce a capture. Then build the single-file cache with the matching reproduce/bd_cache_*.py, which applies the T>=8 generation-length floor.

3. Enroll a profile from the cache (arrays of shape [run, step, layer, dim], keyed task__condition):

   python enroll.py --cache path/to/your_cache.npz --name MyModel --out profiles/MyModel_profile.json
   

4. Monitor new runs against the profile, per task. A deployment knows which task it is running:

   python monitor.py --profile profiles/MyModel_profile.json --cache path/to/your_cache.npz
   

5. Check held-out false positives. Enroll on runs 0-14, score on the untouched runs 15-29:

   python holdout_eval.py
   

holdout_eval.py needs the model caches, which are large and are not shipped. Rebuild them with the reproduce/ scripts, or point SHAPESTATE_CACHE_DIR at them. The tier-2 smoke recipe (smallest model, reduced runs) is in DRY_RUN_LOG.md.

Hardware Requirements

The tier-1 demo is CPU-only and needs only numpy. Everything below is for tier-2 capture and enrollment.

model size	fp16 VRAM (approx)	int8 / int4 VRAM	full-capture disk
1.5-3B	6-8 GB	2-4 GB	~3-5 GB
7-9B	16-20 GB	6-10 GB	~6-10 GB
70B	over one GPU in fp16	~70 GB in 8-bit on one 80 GB GPU	tens of GB

• The int4 and int8 conditions require a CUDA GPU and bitsandbytes. They do not run on CPU.
• Capture stores full hidden states for every run. One model's full grid (four tasks, five conditions, thirty runs) is a few GB on disk for a 1.5-3B model and grows with layers and dimension. Caches and weights stay out of the repo; .gitignore excludes them and the release ships profiles only.
• Capture time is dominated by generation. For a 1.5-3B model the full 600-run grid runs in roughly tens of minutes on an A100-class GPU and grows with model size. These figures are approximate. Time your own with the stage script's per-task timing.
• The seven enrolled models span 1.5B to 9B. A 70B model needs 8-bit loading to fit on a single 80 GB GPU; the 70B row above is capacity guidance for enrolling larger models.

The seven-model laws

The study enrolled seven models across four families (Qwen, Llama, Gemma, Mistral, and Phi as the cold-acceptance model), 1.5B to 9B. Nine structural laws were pre-registered and held across the six-model study (A Qwen2.5-3B, B Llama-3.2-3B, C Gemma-2-2b, D Qwen2.5-1.5B, E Gemma-2-9b, F Mistral-7B):

law	what holds
native-floor match	measured floor within about 5 percent of sqrt(2/(pi d))
cross-task direction present	per-type band cosine 8x to 69x the native floor
int8 depth-conservation	int8 peak depth in the earliest fifth of depth
spectral sign (ordinal)	int4 is the most concentrated corruption at the operating band
task-conditional gate	the off-manifold gate separates within task; a foreign benign task reads off the manifold
low cross-type sharing	per-type residual above 0.5
depth-wise structure	healthy straightening above the random-geometry floor
run-wise correlation absence	noise and int8 distortion at or below the split-half floor
noise tail-visibility	noise is tail-visible at mid-generation; the mean-tail pair is never jointly blind

Two family fingerprints: Gemma is weak-noise and deep-int4. Both are absent on the four non-Gemma models. The cold-acceptance model held eight of the nine laws. The one break was the absolute form of the spectral-sign law, which is why that law is stored in its ordinal form. Full detail is in docs/SUBSTRATE_SIX_MODEL_REPORT.md and docs/BD_G_COLD_ACCEPTANCE.md.

The sensitivity-profile concept

Enrollment produces a per-model, per-class sensitivity profile: which corruption is hardest to detect (the weakest type), where each type's signal lives in depth (its peak layer and band), how far a corrupt batch has to move to flag (the margin and norm thresholds), and the model's native floor and spectral baseline. This is the deployment spec sheet, and it varies widely. The two Gemma models are noise-insensitive, with noise peak margins of 3 and 6, while the others range from 37 to 386. A transplanted threshold would misfire. Enroll per model. Every field is documented in docs/PROFILE_SCHEMA.md.

Scope

The corruption classes tested are global and exogenous: activation noise (sigma 0.01 to 0.05), int4 fp4, int8. Targeted and endogenous corruption (for example LoRA) and adaptive corruption are untested. The nine laws are candidate laws about the tested classes on six models in three families. Each was pre-registered (see prereg/PREREG_LEDGER.md) and could have failed. The four task specs share one domain family. This is a v0. Full detail and limitations are in docs/SUBSTRATE_SIX_MODEL_REPORT.md.

The seven published profiles are demonstrations enrolled on public models. They are not a deployment configuration and do not describe any deployed system's watched loci or blind spots. A real deployment enrolls its own profile from its own model; that production profile is operationally sensitive and should be kept private. The method is public; a specific deployment's enrolled directions and thresholds are what stay secret.

The atlas

The atlas/ directory is a shared table of enrolled profiles, one row per model, with the weakest type, noise sensitivity, and operating-band depth. The seven founding models seed it. To add your own model, enroll it with the recipe and open a pull request. See atlas/ATLAS.md and atlas/SUBMISSION.md. Submitted profiles are licensed CC BY 4.0 with attribution to the submitter, and no CLA is required for a data contribution.

Maintenance

Shapestate is a research release from Tethral, Inc. Issues are welcome and answered on a best-effort basis. Commercial support and licensing are available; see COMMERCIAL.md.

License

The code is under the PolyForm Noncommercial License 1.0.0 (see LICENSE): free for research, experimentation, education, and noncommercial organizations. For commercial use, see COMMERCIAL.md. The research findings documents are under CC BY 4.0 (see LICENSE-docs.md) so they can be quoted and cited freely, including commercially, with attribution. Contributions require the CLA in CONTRIBUTING.md. The seven enrolled profiles derive from third-party models; each source model and its license and attribution are recorded in PROFILES_NOTE.md.

Citation

If you use Shapestate or its findings, cite it via CITATION.cff:

Tethral, Inc. (2026). Shapestate. https://github.com/Tethral-Inc/shapestate

Repository map

• demo.py: the thirty-second CPU-only demo (tier 1).
• enroll.py, monitor.py, holdout_eval.py, bd_margin.py: the v0 tool.
• examples/demo/: the bundled demo dataset (model F) and F's profile.
• profiles/: the seven enrolled model profiles (no weights or caches). See PROFILES_NOTE.md.
• atlas/: the shared profile atlas (ATLAS.md) and the submission guide (SUBMISSION.md).
• reproduce/: the stage scripts that produced the captures and analyses.
• prereg/: the frozen pre-registrations with sha1s and PREREG_LEDGER.md.
• pyproject.toml, ROADMAP.md, docs/LAUNCH_NOTES.md, docs/LAUNCH_DAY_RUNBOOK.md.
• docs/SUBSTRATE_SIX_MODEL_REPORT.md: the technical report.
• docs/BD_SIXMODEL_COOCCURRING.md: the internal-versus-output and collapse-channel analysis.
• docs/BD_G_COLD_ACCEPTANCE.md: the frozen cold-acceptance test on the seventh model.
• docs/RESEARCH_BACKLOG.md: priority-ordered open threads.
• docs/PROFILE_SCHEMA.md: every field of the profile JSON.
• docs/ARXIV_DRAFT.md: the paper-form consolidation of the report.
• LICENSE, LICENSE-docs.md, COMMERCIAL.md, CONTRIBUTING.md, PROFILES_NOTE.md, CITATION.cff, requirements.txt, requirements-demo.txt.