Skip to main content

An extensible config file format

Project description

yay ain't YAML
==============

yay is a configuration file format built on top of YAML based on our experience with the ConfigParser extensions buildout uses. It adds overlays (one config file including and extending another) and variables.

Consider a config file that looks something like this::

foo:
bar:
baz:
- 1
- 2
- 3
qux: wibble.wobble
quux: wobble

goo:
bar:
baz:
- 1
- 2
- 3
- 4
qux: wibble.cobble
quux: cobble

hoo:
bar:
baz:
- 1
- 3
qux: wibble.yobble
quux: yobble

With yay, this might look like this::

foo:
bar:
baz:
- 1
- 2
- 3
qux: wibble.{:quux}
quux: wobble

goo.copy: foo
goo:
bar:
baz.append:
- 4
quux: cobble

hoo.copy: foo
hoo:
bar:
baz.remove:
- 2
quux: yobble

Using yay
---------

To load yay config from a string you can use the 'load' function::

>>> from yay import load
>>> load("""
... foo:
... bar: 1
... """)
{"foo": {"bar": 1}}

If you want to load from a URI (file:// and http:// are supported) then use 'load_uri'::

>>> from yay import load_uri
>>> load_uri("/etc/someconfig.yay")
{"foo": {"bar": 1}}

The return value in both cases is a standard python dictionary.

Changelog for yay
=================

0.0.41 (2011-12-21)
-------------------

- You can inject Python objects into the Yay Config object, as long as Yay
knows how to box them. If you had a Yay file like this::

hello: abc
result: ${hello}

And the following python code::

from yay.config import Config
c = Config()
c.load_uri(example_file)
c.add(dict(hello="xyx"))

Then the following assertion is true::

c.get() == dict(hello="xyz", result="xyz")


0.0.40 (2011-12-13)
-------------------

- Fixes calling macros defined in other files.


0.0.39 (2011-11-30)
-------------------

- Fix string builder so that configs containing secrets can be pickled.


0.0.38 (2011-11-22)
-------------------

- This is a bugfix release to improve existing features, especially boxing of
foreign data.

- Move more code away from direct use of ``Boxed`` to ``BoxingFactory.box``

- Wrap any dictionary objects in a Mapping object so they can be merged with
other mappings that come from within Yay.


0.0.37 (2011-11-20)
-------------------

- Brown paper bag release to remove stray print.


0.0.36 (2011-11-20)
-------------------

- There is now a ``home://`` URL handler. For my laptop this means::

import yay
yay.load_uri("home://foo/bar.yay")

Is equivalent to::

import yay
yay.load_uri("/home/john/foo/bar.yay")

- Improve searchpath handling to better cope with URIs.

- Add support for ``else`` keyword in expressions as per YEP2::

foo:
bar: ${foo.baz else 52}

This is useful for providing sensible defaults in your recipes.

- The loop block is now autoflattening. This means that constructs that yield
lists of lists will be turned into 1-dimensional lists by ``.foreach``. For
example::

somelist:
- - - a
- b
- c
- d

someotherlist.foreach var in somelist: ${var}

The list ``someotherlist`` would now contain::

somelist:
- a
- b
- c
- d

The previous behaviour can be obtained with the ``chain`` modifier::

somelist.foreach var in somelist chain: ${var}

- Add a simple macro language. This is highly EXPERIMENTAL and subject to
frequent syntax tweaks. You can define a reusable block of configuration
using the .define statement. An example of this in Yaybu to create a reusable
virtualenv step::

.define Virtualenv:
- Directory:
name: ${venv.location}
owner: ${venv.owner}
- Execute:
name: create-virtualenv-${venv.location}
command: virtualenv --no-site-packages ${venv.location}
creates: ${venv.location}/bin/pip
- Execute:
name: install-requirements-${venv.location}
command: ${venv.location}/bin/pip install -r ${venv.requirements}

The macro is callable inline as part of an expression. For example::

resources.append:
.foreach venv in virtualenvs: ${Virtualenv!}

Equally you can do this::

.define Virtualenv:
- Directory:
name: ${location}
owner: ${owner}
# SNIP

resources.append:
Virtualenv!:
location: /tmp/example
owner: dave
requirements: /tmp/requirements.txt


0.0.35 (2011-11-12)
-------------------

- This release refactors ProtectedString to make it more useful outside Yay.
Instead of importing ``yay.protectedstring.ProtectedString`` you should
import ``yay.String``

- You can now pass strings to the constructor and they will automatically be
wrapped. Previously you had to pass StringPart's to the constructor. That is
now an internal implementation detail.

- There is now an extend() function. This accepts lists that contains a mixture
of raw strings and other ``String`` objects::

s = String("echo")
y = String("supersekritpassword", secret=True)
s.extend(["a", "b", y, "d"])

- There is now an as_list() function. This returns all the parts you have added
to a string. This is useful if you are building a command line to pass to
subprocess::

s.String(["someprogram", "--pasword"])
s.add_secret("password")

# Log the obfuscated version but execute with the real password
log.info(s.protected)
p = subprocess.Popen(s.as_list(secret=False), cwd="/")

- There is now a ``secret`` call in Yay for exercising secret Yay without
needing to use GPG. This is mainly for test purposes, but might be useful if
you have strings which arent closely guarded secrets yet you dont want them
show in logs. To use it you do something like this::

resources.append:
- Checkout:
name: /checkouts/mycode
repository: http://github.com/whatever
scm_password.secret: mypassword

This example is from Yaybu and would stop Yaybu from logging your SVN
password.


0.0.34 (2011-11-10)
-------------------

- Compose now reuses Opener() rather than creating a new one for each stream
loaded.


0.0.33 (2011-11-10)
-------------------

- You can now pass ``searchpath`` to the ``Config`` object.


0.0.32 (2011-11-10)
-------------------

- Working package:// imports


0.0.31 (2011-11-10)
-------------------

- Better absolute path handling


0.0.30 (2011-11-10)
-------------------

- Any stream objects returned by the ``Openers()`` API now have a ``len`` property.

- The internal ``Openers()`` API can now load from any package on the Python path::

fp = Openers().open("package://some.egg/hello.txt")


0.0.29 (2011-11-10)
-------------------

- The internal ``Openers()`` API now has support for a search path::

fp = Openers(searchpath=['file:///home/john', 'http://google.com']).open("foo.txt")

- The ``Openers()`` API now supports https://


0.0.28 (2011-11-07)
-------------------

- Fully remove all Mapping default crud


0.0.27 (2011-11-03)
-------------------

- Fix ${django.SomeModel} to properly resolve


0.0.26 (2011-11-03)
-------------------

- Add support for a Django DataStore
- Allow objects exposed from Python, including simple method calls


0.0.25 (2011-10-28)
-------------------

- Fix regression in .append


0.0.24 (2011-10-22)
-------------------

- Can now foreach over a mapping
- Add ``.foreach x in y if x.z = a``
- Add '.with expression as foo:'
- Add chain and nochain mode to foreach. chain is the default.
- Major refactoring, context variable is no longer needed to resolve the graph
- The 'semi_resolve' API is now more correctly named 'expand'


0.0.23 (2011-07-26)
-------------------

- $$ escapes $ - so $${foo} is treated as a string, not a variable lookup


0.0.22 (2011-07-19)
-------------------

- Remove spurious debug messages
- Fix .import on a .foreach


0.0.21 (2011-07-19)
-------------------

- New .include directive that can use variables and appear anywhere in file


0.0.20 (2011-06-29)
-------------------

- Change ProtectedString to inherit from basestring


0.0.19 (2011-06-29)
-------------------

- Concept of a protected yay file, where any strings that end up containing
secrets will be obscured
- ProtectedString that handles concatenation of protected and unprocted strings
- GPG used to decrypt .yay.gpg files, any variables they spawn are protected

0.0.18 (2011-06-10)
-------------------

- Brown paper bag to remove stray debugging scaffold


0.0.17 (2011-06-10)
-------------------

- Fix appending to None
- Fix list access where list has already been partially resolved

0.0.16 (2011-06-10)
-------------------

- Add range tests, fixed range() as a ${} expansion
- Added sum() (but no syntax sugar)
- Allow index operations against filters
- Fix iterating over Flatten() nodes
- Fix iterating over ForEach() nodes
- Add test for empty documents that got away

0.0.15 (2011-05-18)
-------------------

- Raise an error if field access is invalid
- Add a LanguageError for displaying helpful messages when yay files
are wrong
- It is now a bug if we raise an error that doesn't descend from
yay.errors.Error


0.0.14 (2011-05-12)
-------------------

- Don't break on empty (e.g. {})


0.0.13 (2011-03-06)
-------------------

- Fix Append nodes


0.0.12 (2011-03-03)
-------------------

- Export that method ;)


0.0.11 (2011-03-03)
-------------------

- Don't be unicode unless needed
- Add a dump() method


0.0.10 (2011-02-22)
-------------------

- Don't chomp whitespace in bracketed_expression


0.0.9 (2011-02-22)
------------------

- Don't chomp whitespace in templated_string


0.0.8 (2011-02-18)
------------------

- Replace 'foreach bar as foo' with 'foreach foo in bar'


0.0.7 (2011-02-16)
------------------

- Numbers starting 0 are treated as base 8.


0.0.6 (2011-02-13)
------------------

- Avoid unicode mapping keys where possible - they break **kwargs in Py2.x


0.0.5 (2011-02-09)
------------------

- Add support for nested foreach

- At least for now, support a ruby style .flatten


0.0.4 (2011-02-04)
------------------

- {foo} can be interpreted as a map by YAML. For now, we will use ${foo} to avoid this

- Function calls to python are implemented - there is now range()

- There is now a foo.select key so switch statements can be used

0.0.3 (2011-01-24)
------------------

- Egg packaging fixes


0.0.2 (2011-01-24)
------------------

- Lots more unittests for expression evaluation and expression parsing

- Drop dependency on OrderedDict

0.0.1
-----

- This is still pre-release, no API or language stability guarantees

- Variable resolving is now done in expression tree without a seperate resolve stage

- Uses pyparsing to parse {foo[bar][@.baz > 5]} type syntax

0.0.0
-----

- This is a pre-alpha release to experiment with what we can and can't do.

- New PyYAML Loader subclass for getting ordered maps without ugly !!omap markup

- Resolves {} markup within YAML nodes

- Lazily evaluates .copy, .append and .remove instructions

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
yay-0.0.41.zip (112.6 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page