Spiral — a reversible looped transformer (~1M core)

One small block, iterated. Each pass folded into a new subspace. Reversible in time. Spiral tests a single hypothesis: a ~1M-parameter block iterated N times can recover the reasoning depth of a ~125M model — online-distilled from a teacher (SmolLM-135M) that is also its baseline. Developed inside Cortex/Dentate; usable standalone.

The idea in one paragraph

A regular transformer stacks D distinct layers. Spiral has one shared block (f_θ, ~1M params) and runs it for N iterations. To stop every iteration doing the same thing (provably weaker than untied depth — Xu & Sato 2024), each pass is folded into a different latent subspace by an orthogonal rotation Q_k and conditioned on the iteration index. The embedded input is re-injected every step (DEQ/Huginn). The momentum coupling makes the augmented state algebraically invertible over the reals under fixed deterministic replay. Floating-point reconstruction is approximate and configuration-dependent. Current training runs a no-gradient prefix and an ordinary-autograd suffix (TBPTT), not reconstruction backward; latent waists, modulation and external memory affect storage. Rotation, weight/gate normalization, QK normalization and z-loss control selected scales but do not certify bounded states or a fixed point. Zero transformer gates mean zero transformer force, not identity of the injected loop. Online knowledge distillation and other objectives are selected by the trainer and recorded in the run configuration.

Install

pip install -e .            # core (torch)
pip install -e '.[train]'   # + transformers/datasets for standalone distillation
pip install -e '.[dev]'     # + pytest/ruff

Use

from spiral import SpiralConfig, build_model
cfg = SpiralConfig(d_model=256, mean_iters=32, reproject="orthogonal", reversible="momentum",
                   teacher_model="smollm-135m")
model = build_model(cfg)            # lazy torch import; moves to cfg.device (mps)
logits = model(input_ids, n_iters=32)
model.num_params(core_only=True)    # 1,023,553

A config roundtrip preserves configuration, not a complete run: reproducibility additionally needs source/runtime, data/reference hashes, seeds, checkpoint/donor artifacts and decoding/masking metadata.

Validate first, trust numbers later

pytest exercises configuration-specific contracts and numerical tolerance checks (see docs/VALIDATION.md). Finite roundtrip, zero-transformer-force, output and gradient checks are useful regressions, not universal precision, stability or nonvanishing-gradient guarantees.

pytest -q

Historical scaffold observations (not independently reproduced for corrected code)

Docs

• docs/REQUIREMENTS.md — F/N spec: architecture, training loop, target, decisions.
• docs/MATH.md — injected/folded/unfolded recurrence, real-arithmetic inverse, actual TBPTT versus proposed reconstruction backward, scale-control limitations and expressivity hypotheses.
• docs/STABILITY.md — full augmented Jacobian, switched-product counterexamples, superseded historical diagnostic outputs and empirical controls without all-depth guarantees.
• docs/SPIRAL_LATENT.md / docs/SPIRAL_MEM.md — the memory variants.
• docs/SOURCES.md — verified references (arXiv IDs), grouped by design question (incl. §C.2 gradient-dynamics and §E.2 memory-variant internals).
• docs/VALIDATION.md — the test/monitor-before-you-trust protocol (reconstruction, gradient health, theory-matches-practice).
• docs/PLAN.md — phased build (P0 scaffold ✅ → P6), risk register.

Canonical-paper headline metrics and figures are historical exploratory summaries, not independently reproduced corrected-protocol results. Source repairs do not validate them retroactively; frozen artifacts, causal/held-out recall controls and common evaluation references are required.

Honesty

~1M coherence needs a narrow distribution (TinyStories 2023); broad-pool 100× knowledge compression is unsupported (capacity gap). Spiral targets narrow per-domain competence first and treats broad-pool 100× as the research hypothesis under test — not a promise. The bet is on reasoning depth, where looping, not parameters, is the bottleneck. License: Apache-2.0.