Skip to main content

Hierarchical Yaml Python Config

Project description

hiyapyco
========

HiYaPyCo - A Hierarchical Yaml Python Config

Description
-----------

A simple python lib allowing hierarchical overlay of config files in
YAML syntax, offering different merge methods and variable interpolation
based on jinja2.

The goal was to have something similar to puppets hiera
``merge_behavior: deeper`` for python.

Key Features
------------

- hierarchical overlay of multiple YAML files
- multiple merge methods for hierarchical YAML files
- variable interpolation using jinja2

Requirements
------------

- PyYAML aka. python-yaml
- Jinja2 aka. python-jinja2
- ordereddict for python2.6 (if you like to use the Ordered Dict Yaml
Loader / Dumper aka. ODYLDo)

Python Version
~~~~~~~~~~~~~~

HiYaPyCo was designed to run on both current major python versions
without changes. Tested versions:

- 2.6
- 2.7
- 3.2
- 3.4

Usage
-----

A simple example:

::

import hiyapyco
conf = hiyapyco.load('yamlfile1' [,'yamlfile2' [,'yamlfile3' [...]]] [,kwargs])
print(hiyapyco.dump(conf))

real life example:
~~~~~~~~~~~~~~~~~~

``yaml1.yaml``:

::

---
first: first element
second: xxx
deep:
k1:
- 1
- 2

``yaml2.yaml``:

::

---
second: again {{ first }}
deep:
k1:
- 4
- 6
k2:
- 3
- 6

load ...

::

>>> import pprint
>>> import hiyapyco
>>> conf = hiyapyco.load('yaml1.yaml', 'yaml2.yaml', method=hiyapyco.METHOD_MERGE, interpolate=True, failonmissingfiles=True)
>>> pprint.PrettyPrinter(indent=4).pprint(conf)
{ 'deep': { 'k1': [1, 2, 4, 6], 'k2': [3, 6]},
'first': u'first element',
'ma': { 'ones': u'12', 'sum': u'22'},
'second': u'again first element'}

real life example using yaml documents as strings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

::

>>> import hiyapyco
>>> y1="""
... yaml: 1
... y:
... y1: abc
... y2: xyz
... """
>>> y2="""
... yaml: 2
... y:
... y2: def
... y3: XYZ
... """
>>> conf = hiyapyco.load([y1, y2], method=hiyapyco.METHOD_MERGE)
>>> print (conf)
OrderedDict([('yaml', 2), ('y', OrderedDict([('y1', 'abc'), ('y2', 'def'), ('y3', 'XYZ')]))])
>>> hiyapyco.dump(conf, default_flow_style=True)
'{yaml: 2, y: {y1: abc, y2: def, y3: XYZ}}\n'

args
~~~~

All ``args`` are handled as *file names* or *yaml documents*. They may
be strings or list of strings.

kwargs
~~~~~~

- ``method``: bit (one of the listed below):

- ``hiyapyco.METHOD_SIMPLE``: replace values (except for lists a
simple merge is performed) (default method)
- ``hiyapyco.METHOD_MERGE``: perform a deep merge

- ``interpolate``: boolean : perform interpolation after the merge
(default: False)

- ``castinterpolated``: boolean : try to perform a *best possible
match* cast for interpolated strings

- ``usedefaultyamlloader``: boolean : force the usage of the default
*PyYAML* loader/dumper instead of *HiYaPyCo*\ s implementation of a
OrderedDict loader/dumper (see: Ordered Dict Yaml Loader / Dumper
aka. ODYLDo) (default: False)

- ``failonmissingfiles``: boolean : fail if a supplied YAML file can
not be found (default: True)

- ``loglevel``: int : loglevel for the hiyapyco logger; should be one
of the valid levels from ``logging``: 'WARN', 'ERROR', 'DEBUG', 'I
NFO', 'WARNING', 'CRITICAL', 'NOTSET' (default: default of
``logging``)

- ``loglevelmissingfiles``: int : one of the valid levels from
``logging``: 'WARN', 'ERROR', 'DEBUG', 'INFO', 'WARNING', 'CRITICAL',
'NOTSET' (default: ``logging.ERROR`` if
``failonmissingfiles = True``, else ``logging.WARN``)

interpolation
~~~~~~~~~~~~~

For using interpolation, I strongly recomend *not* to use the default
PyYAML loader, as it sorts the dict entrys alphabetically, a fact that
may break interpolation in some cases (see ``test/odict.yaml`` and
``test/test_odict.py`` for an example). See Ordered Dict Yaml Loader /
Dumper aka. ODYLDo

default
^^^^^^^

The default jinja2.Environment for the interpolation is

::

hiyapyco.jinja2env = Environment(undefined=Undefined)

This means that undefined vars will be ignored and replaced with a empty
string.

change the jinja2 Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If you like to change the jinja2 Environment used for the interpolation,
set ``hiyapyco.jinja2env`` **before** calling ``hiyapyco.load``!

use jinja2 DebugUndefined
^^^^^^^^^^^^^^^^^^^^^^^^^

If you like to keep the undefined var as string but raise no error, use

::

from jinja2 import Environment, Undefined, DebugUndefined, StrictUndefined
hiyapyco.jinja2env = Environment(undefined=DebugUndefined)

use jinja2 StrictUndefined
^^^^^^^^^^^^^^^^^^^^^^^^^^

If you like to raise a error on undefined vars, use

::

from jinja2 import Environment, Undefined, DebugUndefined, StrictUndefined
hiyapyco.jinja2env = Environment(undefined=StrictUndefined)

This will raise a ``hiyapyco.HiYaPyCoImplementationException`` wrapped
arround the ``jinja2.UndefinedError`` pointing at the string causing the
error.

more informations
^^^^^^^^^^^^^^^^^

See:
`jinja2.Environment <http://jinja.pocoo.org/docs/dev/api/#jinja2.Environment>`__

cast interpolated strings
~~~~~~~~~~~~~~~~~~~~~~~~~

As you must use interpolation as strings (PyYAML will weep if you try to
start a value with ``{{``), you can set ``castinterpolated`` to *True*
in order to try to get a ``best match`` cast for the interpolated
values. **The ``best match`` cast is currently only a q&d implementation
and may not give you the expected results!**

Ordered Dict Yaml Loader / Dumper aka. ODYLDo
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

| This is a simple implementation of a PyYAML loader / dumper using
``OrderedDict`` from collections.
| **Because chaos is fun but order matters on loading dicts from a yaml
file.**
| In order to use this on python 2.6, please install ordereddict:

::

sudo pip-2.6 install ordereddict

Install
-------

From Source
~~~~~~~~~~~

GitHub
^^^^^^

https://github.com/zerwes/hiyapyco

::

git clone https://github.com/zerwes/hiyapyco
cd hiyapyco
sudo python setup.py install

PyPi
^^^^

Download the latest or desired version of the source package from
https://pypi.python.org/pypi/HiYaPyCo. Unpack the archive and install by
executing:

::

sudo python setup.py install

pip
~~~

Install the latest wheel package using:

::

pip install HiYaPyCo

debian packages
~~~~~~~~~~~~~~~

::

echo "deb http://repo.zero-sys.net/hiyapyco/deb ./" > /etc/apt/sources.list.d/hiyapyco.list
gpg --keyserver keys.gnupg.net --recv-key ED7D414C
# or use:
wget http://jwhoisserver.net/key.asc -O - | gpg --import -
gpg --armor --export ED7D414C | apt-key add -
apt-get update
apt-get install python3-hiyapyco python-hiyapyco

rpm packages
~~~~~~~~~~~~

use http://repo.zero-sys.net/hiyapyco/rpm as URL for the yum repo and
http://jwhoisserver.net/key.asc as the URL for the key.

Arch Linux
~~~~~~~~~~

An `AUR package <https://aur.archlinux.org/packages/python-hiyapyco/>`__
is available.

License
-------

(c) 2014 - 2016 Klaus Zerwes `zero-sys.net <https://zero-sys.net>`__

| This package is free software.
| This software is licensed under the terms of the GNU GENERAL PUBLIC
LICENSE version 3 or later, as published by the Free Software
Foundation.
| See https://www.gnu.org/licenses/gpl.html

Changelog
---------

0.4.7
~~~~~

Reverted: logger settings to initial state

Improved: dump

Merged: \* flatten mapping from Chris Petersen geek@ex-nerd.com \* arch
linux package info from Peter Crighton git@petercrighton.de

0.4.6
~~~~~

MERGED: fixes from mmariani

0.4.5
~~~~~

FIXED: issues #9 and #11

0.4.4
~~~~~

deb packages: \* removed support for python 2.6 \* include examples as
doc

0.4.3
~~~~~

FIXED: issue #6 *import of hiyapyco **version** in setup.py causes pip
install failures*

0.4.2
~~~~~

Changed: moved to GPL

Improvements: missing files handling, doc

0.4.1
~~~~~

Implemented: ``castinterpolated``

0.4.0
~~~~~

Implemented: loading yaml docs from string

0.3.2
~~~~~

Improved tests and bool args checks

0.3.0 / 0.3.1
~~~~~~~~~~~~~

Implemented a Ordered Dict Yaml Loader

0.2.0
~~~~~

Fixed unicode handling

0.1.0 / 0.1.1
~~~~~~~~~~~~~

Initial release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

HiYaPyCo-0.4.7.tar.gz (31.2 kB view details)

Uploaded Source

Built Distribution

HiYaPyCo-0.4.7-py2.py3-none-any.whl (13.9 kB view details)

Uploaded Python 2Python 3

File details

Details for the file HiYaPyCo-0.4.7.tar.gz.

File metadata

  • Download URL: HiYaPyCo-0.4.7.tar.gz
  • Upload date:
  • Size: 31.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for HiYaPyCo-0.4.7.tar.gz
Algorithm Hash digest
SHA256 87a2cb2dc45b62a203ceb32f86f4fc0f5ac68851c4e3d7125aaf09ba0ad9d68b
MD5 4af2084dcf3dd4c9fc79f881ee975ac4
BLAKE2b-256 4b0daf1523afea55befcfd44881fe16d53a1d330213f9b540f938b480cddeb1b

See more details on using hashes here.

File details

Details for the file HiYaPyCo-0.4.7-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for HiYaPyCo-0.4.7-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 728bc4b60d6344a5e7f455fed562d33394dafe02e0d16d016c71ee71c5906b88
MD5 e6af67af216cbcbf0263b1d21c223033
BLAKE2b-256 5734769859a1570d689bb04cda4ad95e9d81ce6d70ea3ab12c8e6ece0155bb59

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page