Skip to main content

trainutil

Simple, composable PyTorch training utilities.

  • Batch: declare a batch as a dataclass and collate samples into it.
  • EMA: exponential moving average you can swap into the model for evaluation.
  • Metrics: weighted running means reduced across DDP ranks in one call.

Installation

pip install trainutil

Usage

Batch

@dataclass
class Batch(BaseBatch):
    coords: torch.Tensor
    name: list[str]

a = Batch.from_sample(coords=torch.randn(5, 3), name="a")
b = Batch.from_sample(coords=torch.randn(3, 3), name="b")
batch = Batch.collate([a, b])

batch.coords.shape  # (2, 5, 3)
batch.name          # ['a', 'b']
batch[1]            # Batch of size one
batch.to("cuda")    # every tensor field moves

EMA

ema = EMA(model, decay=0.999)  # averages the parameters with requires_grad

optimizer.step()
ema.update()

with ema.swap():  # model holds the EMA weights inside the block
    validate(model)

With DDP, pass the unwrapped module (model.module). ema.state_dict() uses the same keys as the model, so model.load_state_dict(ckpt["ema"], strict=False) loads the EMA weights for inference.

Metrics

metrics = MetricAccumulator()
metrics.add({"loss": loss, "lddt": lddt}, weight=batch.batch_size)

means = metrics.flush()  # weighted mean over all ranks; call on every rank
if rank == 0:
    wandb.log(means, step=step)

Metadata

Release files for trainutil 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for trainutil 0.2.0
File Size Uploaded Provenance Actions
trainutil-0.2.0.tar.gz 66.9 kB Without provenance Details for trainutil-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for trainutil 0.2.0
File Interpreter ABI Platform Actions
trainutil-0.2.0-py3-none-any.whl Python 3 none any Details for trainutil-0.2.0-py3-none-any.whl

Total release size: 75.2 kB

Release files / trainutil-0.2.0.tar.gz

Download URL trainutil-0.2.0.tar.gz
Size 66.9 kB
Tags Source
SHA-256 checksum
How to use checksums
7898ed38ecc2ec39aaf39766e50d9c0b6ae846871b1a6d633acd891c2b3d6bb2
BLAKE2b-256 checksum
How to use checksums
8a0c813449751b77588b2dcedf849d49cf4d559fd25982de569f67ffb029de33
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.20

Release files / trainutil-0.2.0-py3-none-any.whl

Download URL trainutil-0.2.0-py3-none-any.whl
Size 8.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ec62b2ca291bb09778de31fbddd2859d76a23f75727c2a0f5f2ee2d305e93adf
BLAKE2b-256 checksum
How to use checksums
dc1937bacfc5ac04d2f12ea49f8af238aba4bf84d9b25a8d7c8be79bb5690930
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.20

Security for trainutil 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page