Skip to main content

A YAML based configuration DSL and associated parser/validator.

Project description

# Pyyacc

Python implementation of a Yet Another Configuration Compiler.

## Why? What about INI, XML, JSON, or plain Python?

The YACC DSL provides an efficient way to specify, document, and construct
a typed configuration system that supports common patterns through overlays,
environment injection, default, optional, and required variables.

None of the above support this as simply or clearly as the YACC DSL and
python configuration builder.

## Concepts

The system is built around three concepts:

* A YACC Descriptor, authored in a YAML DSL.
* Overlays which are additive.
* Output, which represents the assembly of a YACC Descriptor and the Overlays.

### YACC Descriptor

The YACC Descriptor borrows heavily from the INI format: a configuration is a
collection of `sections`, with one or more `keys` within a section.

The YACC Descriptor adds the notion of a `Specification`, which describes a `key`:

* The type
* A default value (or `optional` or `required`)
* Documenation
* Examples
* Deprecated (flag)
* Sensitive

#### Example

port: !spec
description: Port to accept general traffic.
type: !!int "0"
value: 80
debug: !spec
description: Is debug mode enabled.
type: !!bool "0"
value: false
public-name: !spec
description: DNS name for routing public traffic (e.g. redirects).
type: !!str ""
value: !environment HOSTNAME
ssl-private-key: !spec
sensitive: true
type: !!str ""
value: !required

### Overlays

An overlay is simply a set of key-level overrides, where last takes precedence.

#### Overlay as JSON

This can be provided as a YAML document (or equivalent JSON):

port: 9999

#### Overlay as ENV variable

Via a key the environment (the prefix is configurable):


### Output

After merging the descriptor and all overlays via the algorithm described below
the compiled configuration is a `Dict[section, Dict[key, value]]`

This can be injected into a running python process or output via the CLL into formats
consumable by other programs (INI, SH, Make, JSON).

## Resolution Algorithm

### Overlay Assembly

desc_object = yml.load(open(file).read()) #: :type: Dict[str, Dict[str, ValueSpec]]

resolver = YaccResolver(desc_object)

#### 1. Parse the descriptor and provided overlays.

The descriptor defines the complete set of accepted parameters as defined by `ValueSpec` objects.

#### 2. Resolve overlays as defined in the environment

default_overlays = [EnvVar("YACC__OVERLAY")]
extra_overlays = os.environ.get("PYACC_RESOLVER__OVERLAYS")
if not extra_overlays:
extra_overlays = default_overlays
return map(yml.load, filter(None, map(read, extra_overlays)))

#### 3. Define the environment key prefix `${ENV_PREFIX}`

The environment prefix is used to resolve key-specific overlays.

env_prefix = os.environ.get("YACC_RESOLVER__ENV_PREFIX", "PYYACC")

#### 4. Construct an overlay from the environment

- Values are parsed as YAML values.
- Section/keys map to:

re.sub(r'[^a-z0-9]', '_', "%s__%s__%s" % (env_prefix, section, key), flags=re.I).upper()

#### 4. Merge down each overlay onto the descriptor.

descriptor.update(*itertools.chain(provided_overlays, extra_overlays, [environment_overlay]))

### Final Resolution


#### 1. Iterate over all ValueSpecs

1. Resolve any values !environment objects.
2. Validate type against the spec.
3. Store values and errors.

## CLI

pyyacc3 -h

Usage: pyyacc3 [options] yaml [yaml ...]

-h, --help show this help message and exit
-v Verbose logging output.
-V Very verbose logging output.
--flat Flatten into 'section.key': value notation
Output format: yaml, json, sh, make are supported.
--no-validate Disable validation [default: on]
Output destination: path where to write output. If not
provided, stdout is used.
Prefix for overlays from the environment
Name of an overlay to load from the environment

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

py-yacc-3.0.1.tar.gz (14.7 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page