Skip to main content

A static configuration parser for python using templates

Project description

TypeConf

A static configuration parser for python using templates

Python is as a dynamic programming language inherently prone to runtime errors. This is especially problematic for long-running programms. A wrong configuration then can lead to the loss of precious computation.

Sounds familiar?

TypeConf builds a configuration parser from templates, that can be hierarchical nested to define your individual configuration. This template can be easily parsed then at the beginning of your code and checked for your individual requirements.

Furthermore, TypeConf helps maintain up-to-date configurations by quickly revealing broken configurations and making easy to support old configurations despite changes.

Installation

From PyPi

pip install typeconf

From source

pip install git+https://github.com/kilsenp/TypeConf.git

Demo

# templates/parent.yaml
attr_int:
    dtype: int
    required: False
    help: "This is an int"
    default: 0
    type: "datatype"
attr_child:
    dtype: child
    required: False
    help: "This a type constructed from another yaml"
    type: "datatype"        
# templates/child.yaml
attr_bool:
    dtype: bool
    required: True
    help: "This is a bool"
    type: "datatype"

TypeConf will be automatically be able to solve the dependencies when building the type.

# main.py
from typeconf import TypeFactory
factory = TypeFactory()
factory.register_search_directory('templates')
template = factory.build_template('parent')

We can now pass a config file to be parsed.

# config.yaml
attr_child:
    attr_bool: True
template.fill_from_file('config.yaml')

This values can also be overwritten by command line arguments, addressing subconfigs through dot separated names.

from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument('task')
# python main.py test attr_child.attr_bool=False
args, unknown_args = parser.parse_known_args()
# args.task = test
template.fill_from_cl(unknown_args)

Finally we create the config that can be used throughout the rest of the code.

config = template.to_config()  # Actual parsing happens here
# {
#    attr_int: 0,
#    attr_child: {
#       attr_bool: False  #overwritten by cli
#    }
# }

Features

  • Static configuration parsing before program is started
  • Easy verification of existing configurations, if they still work with the current pipeline
  • Easy extension of existing configurations by adding default values to templates
  • Automatically make types within a subfolder choosable
  • Comment individual configuration values
  • Overwrite values using the command line or from code
  • Data type testing, ensure the correct datatype:
    • int
    • float
    • str
    • bool

TODO

  • <input type="checkbox" checked="" disabled="" /> clean split between types, attributes, special types
  • <input type="checkbox" disabled="" /> Consistent naming
  • <input type="checkbox" disabled="" /> Allow more combinations e.g. choice of + datatype
  • <input type="checkbox" disabled="" /> better error messages
  • <input type="checkbox" disabled="" /> config from python file
  • <input type="checkbox" disabled="" /> unit tests
  • <input type="checkbox" disabled="" /> @config_file('path_to_cfg')
  • <input type="checkbox" disabled="" /> eval and type are not exclusive. make additional attribute
  • <input type="checkbox" disabled="" /> Better name parser instead of type?
  • <input type="checkbox" disabled="" /> Pretty print with comments
  • <input type="checkbox" checked="" disabled="" /> Command line interface
  • <input type="checkbox" disabled="" /> Conditional requirements. If a is set b also has to be set. Better if b is a part of a? Leads to duplicates
  • <input type="checkbox" disabled="" /> Generation of a seed.
  • <input type="checkbox" disabled="" /> Pip Package
  • <input type="checkbox" disabled="" /> Github Services
  • <input type="checkbox" disabled="" /> Copy From to ensure same training as validation, or make it as default?
  • <input type="checkbox" disabled="" /> ensure two values are equal, but then why even set two?
  • <input type="checkbox" disabled="" /> Config updates, pass multiple configs

Project details


Release history Release notifications | RSS feed

This version

0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for typeconf, version 0.1
Filename, size File type Python version Upload date Hashes
Filename, size typeconf-0.1-py3-none-any.whl (12.1 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size typeconf-0.1.tar.gz (10.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page