Skip to main content

peira-loss

Pure-PyTorch implementation of the PEIRA loss (Arbel, Terver, Ponce, NeurIPS 2026). PEIRA prevents representation collapse in joint-embedding self-supervised learning with the closed-form ridge-regression predictor between the two views, P = Sigma (N + lambda I)^-1, built from exponential moving averages of the views' second moments (EMA rate cosine-annealed from lr to lr_min), plus a ridge lambda on the representations. The moving averages and the step counter live in the module state_dict, so they are saved and restored with a checkpoint. Multi-GPU training works out of the box: batch second moments are summed across ranks with all_reduce.

pip install -e .
import torch
from peira_loss import PEIRALoss

loss_fn = PEIRALoss(latent_dim=256, T_max=total_steps, reg=0.1, lr=0.9, lr_min=0.5)  # ImageNet-1K recipe
z1, z2 = projector(encoder(view1)), projector(encoder(view2))  # [B, 256] each
out = loss_fn(z1, z2)          # dict: loss, peira_loss, trace_P, cse_loss, reg_loss, peira_lr
out["loss"].backward()

predictor_cov selects the Sigma used in P: "batch" (default) uses the current batch's estimate, as in the runs reported in the paper; "ema" uses the moving average, as written in Algorithm 1. N is always the moving average.

Call the loss exactly once per training step: each forward advances the EMA-rate schedule.

Citation

@inproceedings{arbel2026peira,
  title     = {{PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment}},
  author    = {Arbel, Michael and Terver, Basile and Ponce, Jean},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  eprint    = {2605.17671},
  archivePrefix = {arXiv}
}

Metadata

Release files for peira-loss 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for peira-loss 0.1.0
File Size Uploaded Provenance Actions
peira_loss-0.1.0.tar.gz 9.6 kB Without provenance Details for peira_loss-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for peira-loss 0.1.0
File Interpreter ABI Platform Actions
peira_loss-0.1.0-py3-none-any.whl Python 3 none any Details for peira_loss-0.1.0-py3-none-any.whl

Total release size: 17.9 kB

Release files / peira_loss-0.1.0.tar.gz

Download URL peira_loss-0.1.0.tar.gz
Size 9.6 kB
Tags Source
SHA-256 checksum
How to use checksums
780283580ff4ff6f1d8bc1ca6f051ca8b81e09747402d7232570439f6bf483c7
BLAKE2b-256 checksum
How to use checksums
3516c230ca6f892b0089132cc2ae843c4130031b3a8d5fd13493e3d98a5ba8fb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.8

Release files / peira_loss-0.1.0-py3-none-any.whl

Download URL peira_loss-0.1.0-py3-none-any.whl
Size 8.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
36e93e26ac8131aa2e8522232a2e5423b61e0b7ab18d4c15017fca0ff129c534
BLAKE2b-256 checksum
How to use checksums
c6529df14cf91b8761274a8aba16fb6f0420ba745c247c6dec83558c78142a28
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.8

Security for peira-loss 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page