Skip to main content

A struct config parser that you can set up in the

Project description

Espresso Config

A struct config parser that you can set up in the time it takes to make an espresso. To install, run

pip install espresso-config

Python 3.8 or newer is required.

Motivating Example

Imagine you want to run the following experiment:

backbone: t5-large
model:
  metrics:
    rouge:
      _target_: torchmetrics.functional.text.rouge.rouge_score
  tokenizer:
    _target_: transformers.AutoTokenizer.from_pretrained
    pretrained_model_name_or_path: t5-large
  transformer:
    _target_: transformers.AutoModelForSeq2SeqLM.from_pretrained
    max_sequence_length: 64
    pretrained_model_name_or_path: t5-large

Sure, you could parse that yaml file and get a dict. But (a) working with dictionaries is tedious (b) there's no typing, and (c) you don't want to have to declare all blocks each time; it would be good if you could save some commonly used configurations, such as the parameters for one of transformer or tokenizer keys.

Espresso Config allows you to solve all off those problems by specifying a struct class as follows:

from espresso_config import (
    ConfigNode,
    ConfigRegistry,
    ConfigParam,
    ConfigFlexNode
)

@ConfigRegistry.add
class seq2seq(ConfigNode):
    _target_: ConfigParam(str) = 'transformers.AutoModelForSeq2SeqLM.from_pretrained'

@ConfigRegistry.add
class tok(ConfigNode):
    _target_: ConfigParam(str) = 'transformers.AutoTokenizer.from_pretrained'

@ConfigRegistry.add
class rouge(ConfigNode):
    _target_: ConfigParam(str) = 'torchmetrics.functional.text.rouge.rouge_score'

class ApplicationConfig(ConfigNode):
    backbone: ConfigParam(str)
    class model(ConfigNode):
        class transformer(ConfigNode):
            _target_: ConfigParam(str)
            pretrained_model_name_or_path: ConfigParam(str) = '${backbone}'
            max_sequence_length: ConfigParam(int) = 64
        class tokenizer(ConfigNode):
            _target_: ConfigParam(str)
            pretrained_model_name_or_path: ConfigParam(str) = '${backbone}'
        metrics: ConfigParam(ConfigFlexNode) = {}

Then, your YAML configuration can be as simple as:

backbone: t5-large
model:
  transformer@seq2seq: {}
  tokenizer@tok: {}
  metrics:
    rouge@rouge: {}

Voila! To load the config, run:

from espresso_config import config_from_file

config = config_from_file(ApplicationConfig, path_to_yaml)

Placeholder Variable

A placeholder variable is a config value that references another section of the config, e.g. another value or section. It uses syntax ${path.to.key}.

Registry Reference

A registry reference is a reference to a node config that has been added to the config registry. It uses syntax @placeholder_name.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

espresso-config-0.10.1.tar.gz (28.0 kB view hashes)

Uploaded Source

Built Distribution

espresso_config-0.10.1-py3-none-any.whl (31.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page