Skip to main content

A static configuration parser for python using templates

Project description

TypeConf

A static configuration parser for python using templates

Python is as a dynamic programming language inherently prone to runtime errors. This is especially problematic for long-running programms. A wrong configuration then can lead to the loss of precious computation.

Sounds familiar?

TypeConf builds a configuration parser from templates, that can be hierarchical nested to define your individual configuration. This template can be easily parsed then at the beginning of your code and checked for your individual requirements.

Furthermore, TypeConf helps maintain up-to-date configurations by quickly revealing broken configurations and making easy to support old configurations despite changes.

Installation

From PyPi

pip install typeconf

From source

pip install git+https://github.com/kilsenp/TypeConf.git

Demo

# templates/parent.yaml
attr_int:
    dtype: int
    required: False
    help: "This is an int"
    default: 0
    type: "datatype"
attr_child:
    dtype: child
    required: False
    help: "This a type constructed from another yaml"
    type: "datatype"        
# templates/child.yaml
attr_bool:
    dtype: bool
    required: True
    help: "This is a bool"
    type: "datatype"

TypeConf will be automatically be able to solve the dependencies when building the type.

# main.py
from typeconf import TypeFactory
factory = TypeFactory()
factory.register_search_directory('templates')
template = factory.build_template('parent')

We can now pass a config file to be parsed.

# config.yaml
attr_child:
    attr_bool: True
template.fill_from_file('config.yaml')

This values can also be overwritten by command line arguments, addressing subconfigs through dot separated names.

from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument('task')
# python main.py test attr_child.attr_bool=False
args, unknown_args = parser.parse_known_args()
# args.task = test
template.fill_from_cl(unknown_args)

Finally we create the config that can be used throughout the rest of the code.

config = template.to_config()  # Actual parsing happens here
# {
#    attr_int: 0,
#    attr_child: {
#       attr_bool: False  #overwritten by cli
#    }
# }

Features

  • Static configuration parsing before program is started
  • Easy verification of existing configurations, if they still work with the current pipeline
  • Easy extension of existing configurations by adding default values to templates
  • Automatically make types within a subfolder choosable
  • Comment individual configuration values
  • Overwrite values using the command line or from code
  • Data type testing, ensure the correct datatype:
    • int
    • float
    • str
    • bool

TODO

  • [x] clean split between types, attributes, special types
  • [ ] Consistent naming
  • [ ] Allow more combinations e.g. choice of + datatype
  • [ ] better error messages
  • [ ] config from python file
  • [ ] unit tests
  • [ ] @config_file('path_to_cfg')
  • [ ] eval and type are not exclusive. make additional attribute
  • [ ] Better name parser instead of type?
  • [ ] Pretty print with comments
  • [x] Command line interface
  • [ ] Conditional requirements. If a is set b also has to be set. Better if b is a part of a? Leads to duplicates
  • [ ] Generation of a seed.
  • [ ] Pip Package
  • [ ] Github Services
  • [ ] Copy From to ensure same training as validation, or make it as default?
  • [ ] ensure two values are equal, but then why even set two?
  • [ ] Config updates, pass multiple configs

Project details


Release history Release notifications

This version

0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for typeconf, version 0.1
Filename, size File type Python version Upload date Hashes
Filename, size typeconf-0.1-py3-none-any.whl (12.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size typeconf-0.1.tar.gz (10.3 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page