Skip to main content

torch-einops-utils

Some utility functions to help myself (and perhaps others) go faster with ML/AI work

maybe_return

return just the primary output by default, extras only on request

from torch_einops_utils import maybe_return

@maybe_return('memories')
def forward(self, ids, return_memories = False):
    memories = build_memories(ids) if return_memories else None

    return backbone(ids), memories

model(ids)                             # logits
model(ids, return_memories = True)     # Output(logits, memories)
logits, memories = model(ids, return_memories = True)  # unpacks

any number of extras, individually or all at once

@maybe_return('memories', 'hiddens')
def forward(self, ids, return_memories = False, return_hiddens = False):
    ...
    return logits, memories, hiddens

model(ids, return_hiddens = True)  # Output(logits, hiddens)
model(ids, return_all = True)      # Output(logits, memories, hiddens)

primary may be a tuple of names, or a callable receiving the bound call arguments, for when the shape of the primary depends on the mode

def primary_fields(kwargs):
    if not kwargs.get('return_loss', False):
        return ('logits', 'memory') if kwargs.get('return_memory', False) else 'logits'

    return ('loss', 'loss_mask') if not kwargs.get('reduce_loss', False) else 'loss'

@maybe_return('pooled_repr', primary = primary_fields)
def forward(self, ids, return_loss = False, return_memory = False, reduce_loss = False, return_pooled_repr = False):
    ...
    if return_loss:
        return loss, loss_mask, pooled_repr

    return logits, memory, pooled_repr

model(ids)                            # logits
logits, memory = model(ids, return_memory = True)
loss, loss_mask = model(ids, return_loss = True)
loss = model(ids, return_loss = True, reduce_loss = True)
loss, pooled = model(ids, return_loss = True, reduce_loss = True, return_pooled_repr = True)

log_table

clean, aligned logging for a single record or a list of records

from torch_einops_utils import log_table

log_table(ep = 200, reward = 187.04, steps = 1000, avg100 = 66.03)
# ep 200  reward 187.04  steps 1000  avg100 66.03

log_table([
    dict(ep = 200, reward = 187.04, avg100 = 66.03),
    dict(ep = 225, reward = 76.72, avg100 = 89.98),
], formatters = dict(reward = '.2f', avg100 = '.2f'))
ep   reward  avg100
200  187.04   66.03
225   76.72   89.98

formatters takes a format spec or a callable per column, and format_table returns the string without printing

selectors

freeze and grad_scope share one selector syntax, also available directly

from torch_einops_utils import resolve_parameters, resolve_modules

resolve_parameters(model)                        # all
resolve_parameters(model, '*.bias')              # glob over parameter names
resolve_parameters(model, 'encoder.*')
resolve_parameters(model, nn.LayerNorm)          # module type
resolve_parameters(model, model.encoder)         # module instance
resolve_parameters(model, re.compile('head'))    # regex search
resolve_parameters(model, lambda name, p: p.ndim == 1)  # predicate

resolve_modules(model, nn.Linear)                # (name, module) pairs

selectors can be nested in lists, and strict = False skips selectors that match nothing

freeze

freeze and unfreeze parts of a network by any of the selectors above

from torch_einops_utils import freeze, unfreeze, freeze_except, trainable_parameters, frozen

freeze(model)                       # all params
freeze(model, '*.bias')             # glob over param names
freeze(model, 'encoder.*')
freeze(model, nn.LayerNorm)         # by module type
freeze(model, model.encoder)        # by module instance

unfreeze(model, 'encoder.*')
freeze_except(model, 'head.*')      # freeze all but the head

optimizer = AdamW(trainable_parameters(model), lr = 1e-4)

with frozen(model, 'encoder.*'):
    ...                             # frozen only for this block, restored after

with frozen(model, 'encoder.*', eval = True):
    ...                             # also eval, training flags restored on exit

frozen also works as a decorator, on a method or with an explicit module

@frozen
def forward(self, x): ...

@frozen(model)
def evaluate(): ...

grad_scope accepts the same selectors for disable, enable, and hard_set

from torch_einops_utils import grad_scope

with grad_scope(model, disable = 'encoder.*'):
    ...

note: Frozen in the grad utilities is different, it deepcopies a module into a non-training snapshot

Metadata

Release files for torch-einops-utils 0.2.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for torch-einops-utils 0.2.8
File Size Uploaded Provenance Actions
torch_einops_utils-0.2.8.tar.gz 46.9 kB Without provenance Details for torch_einops_utils-0.2.8.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for torch-einops-utils 0.2.8
File Interpreter ABI Platform Actions
torch_einops_utils-0.2.8-py3-none-any.whl Python 3 none any Details for torch_einops_utils-0.2.8-py3-none-any.whl

Total release size: 77.2 kB

Release files / torch_einops_utils-0.2.8.tar.gz

Download URL torch_einops_utils-0.2.8.tar.gz
Size 46.9 kB
Tags Source
SHA-256 checksum
How to use checksums
7ee3b1380aa297e28347163be3eb72b74c56080ad4c16d508b4fa69880eb84fc
BLAKE2b-256 checksum
How to use checksums
a3ddedcf94375919410314f58d78b94ded0acadfeb3a14cf938f17852b366c18
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.17

Release files / torch_einops_utils-0.2.8-py3-none-any.whl

Download URL torch_einops_utils-0.2.8-py3-none-any.whl
Size 30.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6cf6d34e60bbdfe0805f55e7948a8965313625da14e6cdefa15a1bbc9d461194
BLAKE2b-256 checksum
How to use checksums
abbcf7d0705d3ffdf783610e6b3f61784daa8ba23503a6059c3b0c973dedce7f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.17

Security for torch-einops-utils 0.2.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.8 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page