peira-loss
Pure-PyTorch implementation of the PEIRA loss (Arbel, Terver, Ponce, NeurIPS 2026). PEIRA
prevents representation collapse in joint-embedding self-supervised learning with the
closed-form ridge-regression predictor between the two views, P = Sigma (N + lambda I)^-1,
built from exponential moving averages of the views' second moments (EMA rate cosine-annealed
from lr to lr_min), plus a ridge lambda on the representations. The moving averages and
the step counter live in the module state_dict, so they are saved and restored with a
checkpoint. Multi-GPU training works out of the box: batch second moments are summed across
ranks with all_reduce.
Paper: arXiv:2605.17671. Project page: peira-ssl.github.io. Training code, configs and pretrained checkpoints: Basile-Terv/PEIRA.
pip install peira-loss
# or, for development: pip install -e .
import torch
from peira_loss import PEIRALoss
loss_fn = PEIRALoss(latent_dim=256, T_max=total_steps) # Table 1 recipe, ImageNet-1K
# CIFAR-10: PEIRALoss(latent_dim=..., T_max=..., reg=0.7)
z1, z2 = projector(encoder(view1)), projector(encoder(view2)) # [B, 256] each
out = loss_fn(z1, z2) # dict: loss, peira_loss, trace_P, cse_loss, reg_loss, peira_lr
out["loss"].backward()
predictor_cov selects the Sigma used in P: "ema" (default) uses the moving average, as in
Algorithm 1 and the Table 1 runs; "batch" uses the current batch's estimate, as in
the ImageNet-1K ablations of Appendix G.1 (EMA rate 0.9 to 0.5). N is always the moving
average.
Call the loss exactly once per training step: each forward advances the EMA-rate
schedule.
JointSelfDistillationLoss is the self-distillation baseline of the CIFAR-10
initialization-scale experiment (Figure G.7 of the paper). Its predictor is a buffer updated by
an implicit step: call loss_fn.step_predictor() once after each optimizer step.
Citation
@inproceedings{arbel2026peira,
title = {{PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment}},
author = {Arbel, Michael and Terver, Basile and Ponce, Jean},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2026},
eprint = {2605.17671},
archivePrefix = {arXiv}
}
Metadata
Release files for peira-loss 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| peira_loss-0.2.0.tar.gz | 10.2 kB | Without provenance | Details for peira_loss-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| peira_loss-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for peira_loss-0.2.0-py3-none-any.whl |
Total release size: 18.8 kB
Release files / peira_loss-0.2.0.tar.gz
| Download URL | peira_loss-0.2.0.tar.gz |
|---|---|
| Size | 10.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b47dd71a9e314171c6891c620de210ac04e8ec34342de58fa4fcb882d6728e83
|
|
BLAKE2b-256 checksum How to use checksums |
390ad7a71d593f6f5c6014150007caf6edad592d887dbab777a03ad78a5cdd7e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.8
|
Release files / peira_loss-0.2.0-py3-none-any.whl
| Download URL | peira_loss-0.2.0-py3-none-any.whl |
|---|---|
| Size | 8.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5461a677fab19b8b269c54923c4abed4e72c97435b3cb0f65316790d200ba000
|
|
BLAKE2b-256 checksum How to use checksums |
a9771fc59b38b74b29ce196523218d804cd36766df23223d35ddd0d18ee3f2c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.8
|
Security for peira-loss 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.