shtuka 0.2.0

Neat and tidy configs gadget with methods on steroids 🔥

shtuka

(russian: "a thing"): Neat and tidy configs gadget with methods on steroids 🔥

---

Getting Started

pip install shtuka (py3.6+)

TL;DR

"""Basic stuff"""  # flake8: noqa
import torch
import shtuka

@shtuka.mark('data', alias='D')  # W/o explicit key - __name__ will be used.
def get_data(part, source='mnist', image_size=28):
    pass

class CustomResNet(torch.nn.Module):
    def __init__(self, input_size, num_classes=3, dropout: float = 0):
        """Docstring/inline type hinting helps correct parsing for cli."""
        super().__init__()
    
    def fit(self, data, lr):
        pass

    def test(self, data, metrics):
        pass

# Outside marking with alias.
CustomResNet = shtuka.mark.model(CustomResNet)
CustomResNet.test = shtuka.mark.T(CustomResNet.test)

def main(cfg):
    # 'shtuka' automatically propagate cfg.data kwargs to calls.
    train_data, eval_data = get_data('train'), get_data('eval')
    # But you can also add additional kwarg to override it.
    model = CustomResNet(
        # `x_` unwraps shtuka. Passing wrapped one also working for marked obj.
        input_size=cfg.data.image_size.x_,
        # `f_` is finalizing operator and outputs lambda result.
        num_classes=cfg.data.source.f_(lambda x: 10 if x == 'mnist' else 2),
    )

    # Valid lr format could be: string, float, incorrect or missing.
    model.fit(train_data, lr=cfg.optim.lr.coerce_(float) | 1e-3)
    # `fs_` applies lambda to composite structure.
    result = model.test(
        data=eval_data,
        metrics=cfg.test.metrics.fs_(','.join),
    )

    # Multi-dot set doesn't fail w/o 'stats' key. It creates dict on-the-fly.
    cfg.stats.num_weights = model.num_weights
    # Save full instances specification.
    cfg.eval, cfg.model = shtuka.describe(get_data), shtuka.describe(model)

    if cfg.log.save_result:  # If 'save_results' exists and is True.
        print(cfg.log.msg_file.f_('read'))  # There are many such funcs.
        torch.save(
            {cfg.frozen_(): result},  # `cfg.frozen_()` is immutable.
            # '/' chooses right value is 'save_path' is missing.
            cfg.log.save_path / '/data/result.pth',
        )

if __name__ == '__main__':
    # 'validate' checks collision with methods 
    # (like 'f_' as a config key and as a method).
    shtuka.run(main, validate=True)

base.yaml:

optim:
    lr: '4e-3'

T:
    metrics:
        - acc
        - recall

CLI Run:

python train.py -c base.yaml D.image_size=24 T.metrics.0=f1

Features

First Feature

"""Non-basic stuff"""
import shtuka
cfg = shtuka.cook({'kek': {'mem': 0}})
assert cfg.kek.mem0.miss_

Second Feature

"""Non-basic stuff"""
import shtuka
cfg = shtuka.cook({'kek': {'mem': 0}})
assert cfg.kek.mem0.miss_

Development & Contributing

Pick an issue, branch out from master, file a PR and comply with tox.

Format/Lint/Test/Tox

pip install -e '.[dev]'  
make format  # => lint/test or just flt.  
make [clean] tox

Git/Release

Don't forget to update changelog and create GitHub release manually.

make setup-pre-commit
make release VERSION='?.?.?'

License

shtuka is MIT licensed, as could be found in the LICENSE file.