torch-einops-utils
Some utility functions to help myself (and perhaps others) go faster with ML/AI work
maybe_return
return just the primary output by default, extras only on request
from torch_einops_utils import maybe_return
@maybe_return('memories')
def forward(self, ids, return_memories = False):
memories = build_memories(ids) if return_memories else None
return backbone(ids), memories
model(ids) # logits
model(ids, return_memories = True) # Output(logits, memories)
logits, memories = model(ids, return_memories = True) # unpacks
any number of extras, individually or all at once
@maybe_return('memories', 'hiddens')
def forward(self, ids, return_memories = False, return_hiddens = False):
...
return logits, memories, hiddens
model(ids, return_hiddens = True) # Output(logits, hiddens)
model(ids, return_all = True) # Output(logits, memories, hiddens)
primary may be a tuple of names, or a callable receiving the bound call arguments, for when the shape of the primary depends on the mode
def primary_fields(kwargs):
if not kwargs.get('return_loss', False):
return ('logits', 'memory') if kwargs.get('return_memory', False) else 'logits'
return ('loss', 'loss_mask') if not kwargs.get('reduce_loss', False) else 'loss'
@maybe_return('pooled_repr', primary = primary_fields)
def forward(self, ids, return_loss = False, return_memory = False, reduce_loss = False, return_pooled_repr = False):
...
if return_loss:
return loss, loss_mask, pooled_repr
return logits, memory, pooled_repr
model(ids) # logits
logits, memory = model(ids, return_memory = True)
loss, loss_mask = model(ids, return_loss = True)
loss = model(ids, return_loss = True, reduce_loss = True)
loss, pooled = model(ids, return_loss = True, reduce_loss = True, return_pooled_repr = True)
log_table
clean, aligned logging for a single record or a list of records
from torch_einops_utils import log_table
log_table(ep = 200, reward = 187.04, steps = 1000, avg100 = 66.03)
# ep 200 reward 187.04 steps 1000 avg100 66.03
log_table([
dict(ep = 200, reward = 187.04, avg100 = 66.03),
dict(ep = 225, reward = 76.72, avg100 = 89.98),
], formatters = dict(reward = '.2f', avg100 = '.2f'))
ep reward avg100
200 187.04 66.03
225 76.72 89.98
formatters takes a format spec or a callable per column, and format_table returns the string without printing
selectors
freeze and grad_scope share one selector syntax, also available directly
from torch_einops_utils import resolve_parameters, resolve_modules
resolve_parameters(model) # all
resolve_parameters(model, '*.bias') # glob over parameter names
resolve_parameters(model, 'encoder.*')
resolve_parameters(model, nn.LayerNorm) # module type
resolve_parameters(model, model.encoder) # module instance
resolve_parameters(model, re.compile('head')) # regex search
resolve_parameters(model, lambda name, p: p.ndim == 1) # predicate
resolve_modules(model, nn.Linear) # (name, module) pairs
selectors can be nested in lists, and strict = False skips selectors that match nothing
freeze
freeze and unfreeze parts of a network by any of the selectors above
from torch_einops_utils import freeze, unfreeze, freeze_except, trainable_parameters, frozen
freeze(model) # all params
freeze(model, '*.bias') # glob over param names
freeze(model, 'encoder.*')
freeze(model, nn.LayerNorm) # by module type
freeze(model, model.encoder) # by module instance
unfreeze(model, 'encoder.*')
freeze_except(model, 'head.*') # freeze all but the head
optimizer = AdamW(trainable_parameters(model), lr = 1e-4)
with frozen(model, 'encoder.*'):
... # frozen only for this block, restored after
with frozen(model, 'encoder.*', eval = True):
... # also eval, training flags restored on exit
frozen also works as a decorator, on a method or with an explicit module
@frozen
def forward(self, x): ...
@frozen(model)
def evaluate(): ...
grad_scope accepts the same selectors for disable, enable, and hard_set
from torch_einops_utils import grad_scope
with grad_scope(model, disable = 'encoder.*'):
...
note: Frozen in the grad utilities is different, it deepcopies a module into a non-training snapshot
Metadata
Release files for torch-einops-utils 0.2.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| torch_einops_utils-0.2.7.tar.gz | 46.2 kB | Without provenance | Details for torch_einops_utils-0.2.7.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| torch_einops_utils-0.2.7-py3-none-any.whl | Python 3 | none | any | Details for torch_einops_utils-0.2.7-py3-none-any.whl |
Total release size: 75.9 kB
Release files / torch_einops_utils-0.2.7.tar.gz
| Download URL | torch_einops_utils-0.2.7.tar.gz |
|---|---|
| Size | 46.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7589ffe58bc383ef9bfb341e43d7a928091536260b886e7dea6049103bb28b5e
|
|
BLAKE2b-256 checksum How to use checksums |
a9af5bbcabdf0b4c9a40f2a16f00dbb631664c393eb766dd8eb7aa3937374539
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.17
|
Release files / torch_einops_utils-0.2.7-py3-none-any.whl
| Download URL | torch_einops_utils-0.2.7-py3-none-any.whl |
|---|---|
| Size | 29.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a6afcecd75d7190c71540686fd56e593aae8670809c090908d1f988f6f9b49ec
|
|
BLAKE2b-256 checksum How to use checksums |
64d0bf243f143f1f2b99bd4554e7776459e5a6cc8a7bb180ab3abcff2993d94a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.17
|
Security for torch-einops-utils 0.2.7
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.