Nervecode

The active Revision 4 scientific contract is documented in docs/research/r4_research_contract.md and frozen in configs/r4/research_contract_v1.json.

Nervecode is developing a frozen layerwise representation-trace monitor for post-hoc shift and risk diagnostics. The monitor is designed to preserve predictor outputs while fitting ID-only representation models and exposing request-local empirical rank-atypicality scores. See docs/terminology.md for the R4 vocabulary.

R4 is currently a pre-execution falsification protocol, not a completed efficacy result. The repository contains correctness evidence and a passing 16-check mathematical verifier, but near-/far-OOD, predictor-risk, cost, and final-decision claims remain locked until the runtime gate, clean preflight, and bounded pilot complete. Transition and localization claims are inactive. The earlier complementarity hypothesis is retained only as an empirical question for that pilot, not as an established advantage over logit baselines.

The legacy wrapper APIs below are retained for migration only. They cannot produce R4 claim-grade artifacts.

Installation

• Prerequisites: Python 3.10+, PyTorch 2.0+ (install a build matching your platform from pytorch.org).
• From a checkout for local use: pip install -e .
• For development with tooling: pip install -e .[dev] then pre-commit install.
• Optional extras: .[benchmark] for source-checkout benchmark scripts, .[viz] for plotting, .[logging] for richer logs.

For v0.1 compatibility only, nervecode.wrap(...) instruments a model in place and returns a WrappedModel container. It is not the R4 claim path.

R4 Frozen Monitor Quickstart

The R4 interface instruments a frozen predictor, fits every detector component only on declared ID roles, and returns a request-local observation. Raw fitting, rank-reference fitting, and aggregate-tail fitting are deliberately separate stages. It does not add a training loss or modify the predictor.

import torch
from torch import nn
import nervecode as nvc
from nervecode.fitting.base import FitContext

predictor = nn.Sequential(nn.Linear(2, 8), nn.ReLU(), nn.Linear(8, 2))
config = nvc.MonitorConfig.from_dict(
    {
        "root_seed": 7,
        "capture": {"layers": [{"name": "0"}]},
        "reducer": {"output_dim": 2},
        "state_model": {"clusters": 2},
        "aggregation": {"rule": "mean"},
        "calibration": {"alpha": 0.05},
    }
)
monitor = nvc.instrument(predictor, config)

# Each batch list comes from a distinct, hash-recorded ID split role.
probe_batches = [torch.randn(16, 2)]
monitor.fit_probe_from_batches(
    probe_batches, context=FitContext("id_probe_fit", "probe-split-hash")
)
normalization_batches = [torch.randn(16, 2)]
monitor.fit_rank_references(
    normalization_batches,
    role="id_score_normalize",
    split_hash="normalization-split-hash",
    sample_ids=tuple(f"normalization:{i}" for i in range(16)),
)
calibration_batches = [torch.randn(20, 2)]
monitor.fit_aggregate_tail(
    calibration_batches,
    role="id_aggregate_calibrate",
    split_hash="calibration-split-hash",
    sample_ids=tuple(f"calibration:{i}" for i in range(20)),
)
monitor.mark_ready()

logits, observation = monitor.forward_with_observation(torch.randn(4, 2))
assert torch.equal(logits, observation.predictor_output)
print(observation.aggregate_score, observation.aggregate_p_value, observation.alert)

Persist the complete frozen state with monitor.save_probe_bundle(...); loading verifies the predictor and declared layer signatures. See docs/quickstart.md, the R4 score schemas, and the legacy artifact policy.

Legacy wrapper migration example

The following code uses the v0.1 wrapper API. It remains available for compatibility and labelled exploratory work, but it is not the R4 primary monitor path and must not produce R4 claim artifacts.

Legacy v0.1 wrapper details (compatibility only)

Everything in this section is historical wrapper material. It may be useful for reproducing v0.1 exploratory work, but it must not be used for v0.2 claim artifacts. The frozen-monitor quickstart above is the current public path.

Product Boundary

• Is: a lightweight PyTorch library that wraps selected layers (start with Linear), learns codebooks over reduced activations, and emits layer-wise and aggregated surprise scores.
• Is not: a hardware project, a full observability platform, or a framework-agnostic toolkit; MVP targets PyTorch only and focuses on observe-only wrappers with modest overhead.

First Public API Shape (MVP)

The initial public surface is intentionally small and convenient:

import nervecode

model = MyModel()
wrapped = nervecode.wrap(model, layers="all_linear")

for x, y in train_loader:
    logits = wrapped(x)
    loss = task_loss_fn(logits, y) + wrapped.coding_loss()
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

wrapped.calibrate(calib_loader)

logits = wrapped(x_test)
surprise = wrapped.surprise()  # includes score and percentile

# Optional explicit trace path for robust integrations
logits, trace = wrapped.forward_with_trace(x_test)

Provisional API entries:

• wrap(...)
• WrappedModel.coding_loss()
• WrappedModel.calibrate(...)
• WrappedModel.surprise()
• WrappedModel.forward_with_trace(...)

MVP Scope (legacy exploratory)

This wrapper-based, learned-projection path is retained only for v0.1 compatibility and exploratory joint training. It is not a frozen-monitor or claim-grade R4 configuration.

The MVP is a narrow, end-to-end vertical slice:

• Gradient-updated codebooks and differentiable soft assignment.
• SoftCode and CodingTrace data structures.
• CodingLinear wrapper and wrap(..., layers="all_linear") convenience. The wrapper supports an optional coding_dim to project wide layer outputs down to a coding space via a learned linear reducer while preserving the layer's visible output.
• Mean and max aggregation for a per-input surprise score.
• Empirical percentile calibration on in-distribution data.
• Lightweight coding loss and basic diagnostics (CSV/JSONL).
• One small end-to-end example (MLP or simple CNN).

Distance-augmented surprise:

• The combined per-position surprise can include a distance component to lift OOD scores above ID across the bulk, improving percentile thresholding. Set assignment.beta_distance > 0 (e.g., 0.2–1.0) to enable S = βL·L + βH·H + βD·D where D ≈ log1p(nearest-center squared distance).

The former wrapper quickstarts have been replaced by the frozen-monitor examples. For the current public workflow, use examples/quickstart_mlp.py, examples/quickstart_cnn.py, and docs/quickstart.md.

For a fast, dataset-agnostic smoke run suitable for CI or local validation, use scripts/train_minimal.py which trains a tiny model on a synthetic dataset and calibrates an empirical percentile threshold.

For a minimal OOD benchmark harness, see benchmarks/ood/simple.py which trains an MLP, calibrates percentiles on in-distribution data, and reports AUROC versus a synthetic OOD split.

Benchmark scripts are source-checkout research tools, not part of the installed wheel API. Run them from a checkout with benchmark dependencies installed: pip install -e '.[benchmark,viz]'.

Reusable v0.1.1 benchmark configs live under configs/v0_1_1/; the CIFAR runner accepts them with --config.

The CIFAR ResNet-18 runner has explicit presets:

python -m benchmarks.ood.cifar_resnet18 --preset demo --limit-eval 256
python -m benchmarks.ood.cifar_resnet18 --preset full_cifar10_resnet18 --device cuda

demo is a short smoke path and writes a warning in the run directory; do not use its tables as benchmark claims. full_cifar10_resnet18 uses from-scratch CIFAR-10 defaults, saves per-seed checkpoints by default, records checkpoint SHA-256 values in meta.json, and blocks OOD reporting unless the CIFAR-10 test accuracy gate passes. The runner does not enable ImageNet-pretrained transfer; add and label any future transfer path separately as transfer_resnet18.

For CIFAR OOD combo ablations, the headline-safe method is combo_id_zscore_equal: Energy, Mahalanobis, and Nervecode scores are normalized on ID calibration data only and averaged with fixed equal weights. See docs/combo.md. Learned OOD-calibrated combo rows are oracle appendix results, not headline detector claims.

The CIFAR OOD benchmark also writes real per-sample, per-layer Nervecode surprise artifacts under runs/<run_id>/layerwise/seed_<seed>/*.npz. Plot them without rerunning the model:

python -m nervecode.viz.layerwise runs/<run_id> \
  --seed 123 \
  --ood dtd,cifar100,svhn \
  --output runs/<run_id>/plots/layerwise.png

For quick ablations over codebook/coding hyperparameters and layer selection, use scripts/ablate_grid.py which sweeps small grids of K (codebook size), D (coding dimension), T (temperature), and selection strategies, then logs a minimal quality metric and overhead proxies to CSV.

For a minimal OOD comparison using synthetic scores and the empirical percentile calibrator, see examples/ood_smoke_test.py.

Performance notes: see docs/overhead.md for pooled Conv2d coding overhead estimates, timing harness, and operating guidance.

Recommended OOD Settings (legacy wrapper quick start)

• Selection: layers=first_linear
• Aggregation: agg=max
• Coding: coding_dim D=8
• Codebook: K=16
• Weights: βL=1.0, βE=1.0, βD=1.0 (distance-augmented surprise)
• Calibration: quantile q=0.90 (use 0.95 for stricter ID control)

Run the bundled OOD benchmark with these settings:

python -m benchmarks.ood.simple --epochs 20 --device cpu \
  --agg max --layers first_linear --K 16 --coding-dim 8 \
  --beta-length 1.0 --beta-entropy 1.0 --beta-distance 1.0 \
  --quantile 0.90 --json

Or sweep a narrow fast grid:

FAST=1 bash scripts/run_ood_matrix.sh

Contributing

Contributions are welcome. Please see CONTRIBUTING.md for a quick start, coding guidelines, and how to run tests locally.

Changelog

User-facing changes are tracked in CHANGELOG.md under the Unreleased section and versioned entries.