Skip to main content

peira-loss

Pure-PyTorch implementation of the PEIRA loss (Arbel, Terver, Ponce, NeurIPS 2026). PEIRA prevents representation collapse in joint-embedding self-supervised learning with the closed-form ridge-regression predictor between the two views, P = Sigma (N + lambda I)^-1, built from exponential moving averages of the views' second moments (EMA rate cosine-annealed from lr to lr_min), plus a ridge lambda on the representations. The moving averages and the step counter live in the module state_dict, so they are saved and restored with a checkpoint. Multi-GPU training works out of the box: batch second moments are summed across ranks with all_reduce.

Paper: arXiv:2605.17671. Project page: peira-ssl.github.io. Training code, configs and pretrained checkpoints: Basile-Terv/PEIRA.

pip install peira-loss
# or, for development: pip install -e .
import torch
from peira_loss import PEIRALoss

loss_fn = PEIRALoss(latent_dim=256, T_max=total_steps)  # Table 1 recipe, ImageNet-1K
# CIFAR-10: PEIRALoss(latent_dim=..., T_max=..., reg=0.7)
z1, z2 = projector(encoder(view1)), projector(encoder(view2))  # [B, 256] each
out = loss_fn(z1, z2)          # dict: loss, peira_loss, trace_P, cse_loss, reg_loss, peira_lr
out["loss"].backward()

predictor_cov selects the Sigma used in P: "ema" (default) uses the moving average, as in Algorithm 1 and the Table 1 runs; "batch" uses the current batch's estimate, as in the ImageNet-1K ablations of Appendix G.1 (EMA rate 0.9 to 0.5). N is always the moving average.

Call the loss exactly once per training step: each forward advances the EMA-rate schedule.

JointSelfDistillationLoss is the self-distillation baseline of the CIFAR-10 initialization-scale experiment (Figure G.7 of the paper). Its predictor is a buffer updated by an implicit step: call loss_fn.step_predictor() once after each optimizer step.

Citation

@inproceedings{arbel2026peira,
  title     = {{PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment}},
  author    = {Arbel, Michael and Terver, Basile and Ponce, Jean},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  eprint    = {2605.17671},
  archivePrefix = {arXiv}
}

Metadata

Release files for peira-loss 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for peira-loss 0.2.0
File Size Uploaded Provenance Actions
peira_loss-0.2.0.tar.gz 10.2 kB Without provenance Details for peira_loss-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for peira-loss 0.2.0
File Interpreter ABI Platform Actions
peira_loss-0.2.0-py3-none-any.whl Python 3 none any Details for peira_loss-0.2.0-py3-none-any.whl

Total release size: 18.8 kB

Release files / peira_loss-0.2.0.tar.gz

Download URL peira_loss-0.2.0.tar.gz
Size 10.2 kB
Tags Source
SHA-256 checksum
How to use checksums
b47dd71a9e314171c6891c620de210ac04e8ec34342de58fa4fcb882d6728e83
BLAKE2b-256 checksum
How to use checksums
390ad7a71d593f6f5c6014150007caf6edad592d887dbab777a03ad78a5cdd7e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.8

Release files / peira_loss-0.2.0-py3-none-any.whl

Download URL peira_loss-0.2.0-py3-none-any.whl
Size 8.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5461a677fab19b8b269c54923c4abed4e72c97435b3cb0f65316790d200ba000
BLAKE2b-256 checksum
How to use checksums
a9771fc59b38b74b29ce196523218d804cd36766df23223d35ddd0d18ee3f2c3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.8

Security for peira-loss 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page