ayna
آینه / ayna — mirror. Persian āyeneh; ayna in Turkish and Azerbaijani.
Know whether your inferred dynamics are real, before you interpret them.
ayna reads the outputs of a fitted data-driven dynamics model (LFADS, a sequential autoencoder, a neural ODE, anything that turns spikes into latents and rates) and reports which failure mode it has, and where. It does not train anything. It runs on a CPU in seconds.
pip install ayna
Python 3.10+. Apache 2.0. Three dependencies: numpy, scipy, scikit-learn. No torch, no GPU, no downloads.
Install · Testing · Validation · Roadmap · Publishing · Changelog
The failure that matters
A model can reconstruct held-out neurons almost perfectly and still give you the wrong dynamics. The Computation-through-Dynamics Toolkit (CtD; Versteeg et al., PLOS Comput Biol 2026) names three ways this happens:
- Underfitting — the model misses latent structure that is really there.
- Feature invention — the model adds latent structure that is not there. The readout ignores it, so it costs nothing in reconstruction, and models that invent features often win on co-BPS.
- Dynamical misattribution — the inferred inputs do the work the dynamics should do. Latents and rates look right. Only the inputs, and the dynamics left behind, are wrong.
CtD measures all three when ground truth is known. On a real recording there is no ground truth, and reconstruction alone points the wrong way. ayna runs every check your outputs allow, says which ones it could not run, and gets sharper when you give it more.
How much to trust it
ayna has been tested on three sets of trained models. The tables, and every miss, are in VALIDATION.md.
CtD's 12 pretrained 3BFF models (0.2).
- NODE sweep: every verdict agrees with ground truth.
selectpicks NODE-8 and flags NODE-64 as a reconstruction trap.- LFADS sweep: ayna flags the three low-KL models CtD's paper describes as misattributed.
The 0.2 detectors were redesigned on these models, so this is not a held-out test.
15 held-out models (0.3), trained for this purpose on RandomTarget, MultiTask and a regenerated 3BFF dataset. No threshold was changed after seeing them. Their failures drove 0.4.
12 fresh models for a pre-registered test of 0.4. These are RandomTarget and 3BFF NODE models with new seeds. The expectations were committed before training, and two of the four criteria failed.
| detector | held up | failed |
|---|---|---|
FEATURE_INVENTION |
never wrong when it fires (27 held-out models) | misses invention the readout can see |
SEED_INCONSISTENCY (new) |
never wrong when it fires; 5 of 6 invention cases; caught RandomTarget NODE-10 seed 2, which cycle-consistency misses | missed NODE-10 seed 3 at 0.91; needs two or more seeds |
UNDERFITTING |
cross-fitting fixed RandomTarget's low power (z 1.8 → 9.5) | absolute, not sweep-relative: flags every 3BFF model, which is true below Rate R² 0.90 but not useful for choosing; blind to errors other neurons cannot predict |
INPUT_INHERITED_STRUCTURE |
correct whenever the latents are not invented | misses models whose latents are mostly invented |
DYNAMICAL_MISATTRIBUTION |
CtD's 3BFF LFADS sweep | no working detector for closed-loop tasks: declare closed_loop to stop tier-0 false alarms, but then nothing flags the misattributed model |
Along the way: ayna found that CtD's co-BPS counted 47 held-in neurons as held-out. It was reported in CtD #16 and fixed in PR #18.
Quickstart
# 1. A model output with known, deliberately injected faults
ayna synth field --faults feature_invention misattribution=0.8
# 2. Diagnose it -- no ground truth used for the verdicts
ayna scan datasets/field.npz --out reports/field
# 3. A latent-size sweep: is the best co-BPS model the one to trust?
ayna synth sweep --sweep
ayna select datasets/sweep --out reports/sweep --plot
# 4. How often are the detectors right? (compares against ground truth)
ayna validate
From Python:
from ayna import load_bundle, scan
report = scan(load_bundle("run.npz"))
for f in report["findings"]:
if f["tier"] < 2: # tier 2 = ground-truth confirmations
print(f["severity"], f["id"], f["where"])
scan exits with status 1 when a finding from the model's outputs is an
error, so it can gate a pipeline.
On a CtD sweep, in place
Inside the CtD environment (see INSTALL.md):
W=$HOME_DIR/examples/walkthrough_models
ayna ctd $W/node_sweep --model-type SAE --tt $W/tt_3bff --out reports/node
Each run is loaded in its own subprocess, scanned with its real dynamics attached (tier 1), compared across the sweep, and validated against the teacher.
Three tiers of evidence
| Tier | You provide | Unlocks |
|---|---|---|
| 0 | spikes, predicted rates, latents (+ held-out neurons, + inferred or supplied inputs) | every detector marked tier 0 |
| 1 | a dynamics callable f(z, u) -> z_next (Python API, or automatic with ayna ctd) |
autonomous-rollout and zero-input fixed-point probes |
| 2 | true latents / rates / inputs (synthetic data only) | CtD's Rate R², State R², Input R², and validation |
The detectors
| Finding | Question it answers | Tier |
|---|---|---|
UNDERFITTING |
Do other neurons' spikes still improve the model's predictions (recoverable information, net of a trial-shuffled control, significant across trials)? Does co-BPS fail to beat a smoothed-spikes GLM? | 0 |
FEATURE_INVENTION |
Can the latents be regenerated from the model's own rates (cycle-consistency)? Reports which latent PCs cannot. Comparative only for models with a readout bottleneck (LFADS factors). | 0 |
READOUT_NULLSPACE |
Are there latent directions the readout ignores, where invention could hide later? | 0 |
NONLINEAR_READOUT |
Is the latent→log-rate map far from linear? If so, a nonlinear cycle-consistency replaces the linear one. | 0 |
DYNAMICAL_MISATTRIBUTION (experimental) |
Does the state before each inferred input predict it (lag-1 leakage)? With inferred inputs clamped to zero, do the model's own dynamics lose a state that simply holding would keep? Not reliable in closed-loop tasks. | 0 / 1 |
INPUT_INHERITED_STRUCTURE |
With supplied inputs, how much of the latent variance is just a short filter of the inputs? | 0 |
RECONSTRUCTION_TRAP |
In a sweep, is the best-reconstructing model off the reconstruction × simplicity knee? (ayna select) |
0 |
MISATTRIBUTION_EXCLUDED |
In a sweep, which models must not be recommended however well they reconstruct? (ayna select) |
0 |
SEED_INCONSISTENCY |
In a sweep with several seeds per configuration, can each model's latents be mapped affinely onto a sibling seed's (best R² ≥ 0.90)? Catches invention the readout can see, which cycle-consistency misses. (ayna select) |
0 |
The misattribution probes are ayna's own. CtD lists a ground-truth-free
input-accuracy metric as future work. In closed-loop tasks, where sensory
feedback depends on the model's output, true inputs legitimately depend on
the preceding state. On the held-out RandomTarget LFADS models, tier-0
leakage flagged a model whose inputs were well inferred. Declare such a task
with meta["closed_loop"] = True (or --closed-loop on ayna scan and
ayna ctd): leakage is then reported as information only, and select does
not exclude the model. ayna has no working misattribution detector for
closed-loop tasks yet; see VALIDATION.md.
The bundle
One .npz per fitted model. Rates are expected counts per bin.
| key | shape | |
|---|---|---|
spikes |
(R, T, N) | required |
rates |
(R, T, N) | required: predicted, held-in and held-out |
latents |
(R, T, D) | required |
heldout |
(N,) bool | neurons never given to the model (enables co-BPS) |
val |
(R,) bool | validation trials (default: last 20%) |
trial_lens |
(R,) int | for padded trials |
inputs |
(R, T, U) | with meta["inputs_kind"] = supplied or inferred |
true_latents, true_rates, true_inputs |
ground truth (tier 2) | |
true_inputs_noiseless |
noiseless inputs, preferred as misattribution truth | |
meta |
JSON string | name, model, bin size, readout_bottleneck, closed_loop, anything |
from ayna import Bundle
Bundle(spikes=y, rates=r, latents=z, heldout=ho, inputs=u_hat,
meta={"inputs_kind": "inferred", "name": "lfads_run7", "readout_bottleneck": 8}).save("run7.npz")
report = scan(load_bundle("run7.npz", dynamics=lambda z, u: my_model.step(z, u))) # tier 1
Provenance
pip install "ayna[provenance]"
ayna scan run.npz --out reports/run --daftar
Each scan becomes a daftar run that records the bundle hash, the thresholds, the environment and the headline metrics.
Thresholds
Every threshold has a name and can be overridden
(ayna scan run.npz --set leakage_r2=0.6). The defaults were set on ayna's
synthetic calibration corpus. VALIDATION.md records how each
one was chosen and how it behaved on trained models. Before trusting a
single scan's verdict, compare models of the same class on the same data
with ayna select.
What ayna does not do
- It does not train, fit or tune a model.
- It cannot prove a model's dynamics are right. It can only rule out specific ways they are wrong.
- No DSA or Lyapunov comparison yet.
- Its linear metrics assume the linear-exponential observation model.
NONLINEAR_READOUTtells you when that fails.
Citation
ayna ports metrics and failure-mode definitions from:
Versteeg C, McCart JD, Ostrow M, et al. (2026) Computation-through-Dynamics Toolkit: Simulated datasets and quality metrics for dynamical models of neural activity. PLOS Comput Biol 22(10): e1014755. https://doi.org/10.1371/journal.pcbi.1014755
If you use ayna, please cite that paper.
Related
Part of a small family of pre-flight tools by the same author: daftar (provenance), raftaar (robot demonstration datasets), miras (identifiability of cultural-evolution designs), tarjuman (NeuroML → Jaxley) and baghban (Bonsai workflows).
Research prototype, alpha. Apache 2.0.
Metadata
Release files for ayna 0.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ayna-0.4.0.tar.gz | 104.2 kB | Without provenance | Details for ayna-0.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ayna-0.4.0-py3-none-any.whl | Python 3 | none | any | Details for ayna-0.4.0-py3-none-any.whl |
Total release size: 165.1 kB
Release files / ayna-0.4.0.tar.gz
| Download URL | ayna-0.4.0.tar.gz |
|---|---|
| Size | 104.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
cb92e1728d3e5eb0a84ec267dba9e359608d019d8052da3f4f3432ab04feda5b
|
|
BLAKE2b-256 checksum How to use checksums |
be035ceb741b4523acd1c346a2360da7cab73f2831fd7caf634d1d7260e248d5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.5
|
Release files / ayna-0.4.0-py3-none-any.whl
| Download URL | ayna-0.4.0-py3-none-any.whl |
|---|---|
| Size | 60.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fe952fda1e05e6ba0bba1078cd90e4dc2fa8590b658d7e7cd3e5da6842c87b09
|
|
BLAKE2b-256 checksum How to use checksums |
63b9fec2b5e4fdd4e544f79270ae6a190cfc010624b3d0aaa86c690ef7b06efb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.5
|
Security for ayna 0.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.