ruamel.yaml

ruamel.yaml is a YAML package for Python. It is a derivative of Kirill Simonov’s PyYAML 3.11 which supports YAML1.1

Major differences with PyYAML 3.11:

intergrated Python 2 and 3 sources, running on Python 2.6, 2.7 (CPython, PyPy), 3.3 and 3.4.
round trip mode that includes comments (block mode, key ordering kept)
support for simple lists as mapping keys by transformation to tuples
!!omap generates ordereddict (C) on Python 2, collections.OrderedDict on Python 3, and !!omap is generated for these types.
some YAML 1.2 enhancements (0o octal prefix, \/ escape)
pep8 compliance
tox and py.test based testing
Tests whether the C yaml library is installed as well as the header files. That library doesn’t generate CommentTokens, so it cannot be used to do round trip editing on comments. It can be used for speeded up normal processing (so you don’t need to install ruamel.yaml and PyYaml). See the section Optional requirements.
Basic support for multiline strings with preserved newlines and chomping ( ‘|’, ‘|+’, ‘|-’ ). As this subclasses the string type the information is lost on reassignment. (This might be changed in the future so that the preservation/folding/chomping is part of the parent container, like comments).
RoundTrip preservation of flow style sequences ( ‘a: b, c, d’) (based on request and test by Anthony Sottile)
adding/replacing of comments on block style sequences and mappings with smart column positioning
collection objects (when read in via RoundTripParser) have an lc property that contains line and column info lc.line and lc.col

Round trip including comments

The major motivation for this fork is the round-trip capability for comments. The integration of the sources was just an initial step to make this easier.

adding/replacing comments

Starting with version 0.8, you can add/replace comments on block style collections (mappings/sequences resuting in Python dict/list). The basic for for this is:

from __future__ import print_function

import ruamel.yaml

inp = """\
abc:
  - a     # comment 1
xyz:
  a: 1    # comment 2
  b: 2
  c: 3
  d: 4
  e: 5
  f: 6 # comment 3
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
data['abc'].append('b')
data['abc'].yaml_add_eol_comment('comment 4', 1)  # takes column of comment 1
data['xyz'].yaml_add_eol_comment('comment 5', 'c')  # takes column of comment 2
data['xyz'].yaml_add_eol_comment('comment 6', 'e')  # takes column of comment 3
data['xyz'].yaml_add_eol_comment('comment 7', 'd', column=20)

print(ruamel.yaml.dump(data, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in:

abc:
- a       # comment 1
- b       # comment 4
xyz:
  a: 1    # comment 2
  b: 2
  c: 3    # comment 5
  d: 4              # comment 7
  e: 5 # comment 6
  f: 6 # comment 3

If the comment doesn’t start with ‘#’, this will be added. The key is is the element index for list, the actual key for dictionaries. As can be seen from the example, the column to choose for a comment is derived from the previous, next or preceding comment column (picking the first one found).

Config file formats

There are only a few configuration file formats that are easily readable, and editable: JSON, INI/ConfigParser, YAML (XML is to cluttered to be called easily readable).

Unfortunately JSON doesn’t support comments, and although there are some solutions with pre-processed filtering of comments, there are no libraries that support round trip updating of such commented files.

INI files support comments, and the excellent ConfigObj library by Foord and Larosa even supports round trip editing with comment preservation, nesting of sections and limited lists (within a value). Retrieval of particular value format is explicit (and extensible).

YAML has basic mapping and sequence structures as well support for ordered mappings and sets. It supports scalars are of various types including dates and datetimes (missing in JSON) as a list of YAML has comments, but these are normally thrown away.

Block structured YAML is a clean and very human readable format. By extending the Python YAML parser to support round trip preservation of comments, it makes YAML a very good choice for configuration files that are human readable and editable while at the same time interpretable and modifiable by a program.

Extending

There are normally 6 files involved when extending the roundtrip capabilities: the reader, parser, composer and constructor to go from YAML to Python and the resolver, representer, serializer and emitter to go the other way.

Extending involves keeping extra data around for the next process step, eventuallly resulting in a different Python object (subclass or alternative), that should behave like the original, but on the way from Python to YAML generates the original (or at least something much closer).

Smartening

When you use round-tripping, then the complex data you get are already subclasses of the build in types. So you can patch in extra methods or override existing ones. Some methods are already included and you can do:

yaml_str = """\
a:
- b:
  c: 42
- d:
    f: 196
  e:
    g: 3.14
"""


data = yaml.load(yaml_str, Loader=yaml.RoundTripLoader)

assert data.mlget(['a', 1, 'd', 'f'], list_ok=True) == 196

Examples

Basic round trip of parsing YAML to Python objects, modifying and generating YAML:

from __future__ import print_function

import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

code = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
code['name']['given'] = 'Bob'

print(ruamel.yaml.dump(code, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

Optional requirements

If you have the C yaml library and headers installed, as well as the header files for your Python executables then you can use the non-roundtrip but faster C loader en emitter.

On Debian systems you should use:

sudo apt-get install libyaml-dev python-dev python3-dev

you can leave out python3-dev if you don’t use python3

For CentOS (7) based systems you should do:

sudo yum install libyaml-devel python-devel

Testing

Testing is done using the tox, which uses virtualenv and pytest.

yaml utlity

A utility name yaml is included and allows for basic operations on files:

yaml round-trip <file_name> for basic roundtrip testing of YAML files
yaml json <file_name> for conversion of JSON file(s) to a single YAML block style document
yaml ini <file_name> for conversion of an INI/config file (ConfigObj comment and nested sections supported) to a YAML block style document. This requires configobj to be installed (pip install configobj)
yaml html <file_name> for conversion of the basic structure in a YAML file to a a table in an HTML file. The YAML file:
```
title:
- fruit
- legume
local:
- apple
- sprouts
import:
- orange
- broccoli
```
is converted into the table:

title

fruit

legume

local

apple

sprouts

import

orange

broccoli

See yaml --help for more information on the availble commands

Release files for ruamel.yaml 0.9.7

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ruamel.yaml 0.9.7
File	Size	Uploaded
ruamel.yaml-0.9.7.tar.gz	171.0 kB	Jun 13, 2015	Details

Built distributions (wheels)

Table of built distributions (wheels) for ruamel.yaml 0.9.7
File	Interpreter	ABI	Platform	Reset
ruamel.yaml-0.9.7-cp34-none-win_amd64.whl 72.3 kB Jun 13, 2015	CPython 3.4	none	Windows x86-64	Details
ruamel.yaml-0.9.7-cp34-none-win32.whl 72.3 kB Jun 13, 2015	CPython 3.4	none	Windows x86-32	Details
ruamel.yaml-0.9.7-cp33-none-win_amd64.whl 72.3 kB Jun 13, 2015	CPython 3.3	none	Windows x86-64	Details
ruamel.yaml-0.9.7-cp33-none-win32.whl 72.3 kB Jun 13, 2015	CPython 3.3	none	Windows x86-32	Details
ruamel.yaml-0.9.7-cp27-none-win_amd64.whl 72.4 kB Jun 13, 2015	CPython 2.7	none	Windows x86-64	Details
ruamel.yaml-0.9.7-cp27-none-win32.whl 140.6 kB Jun 13, 2015	CPython 2.7	none	Windows x86-32	Details
ruamel.yaml-0.9.7-cp26-none-win_amd64.whl 72.4 kB Jun 13, 2015	CPython 2.6	none	Windows x86-64	Details
ruamel.yaml-0.9.7-cp26-none-win32.whl 72.4 kB Jun 13, 2015	CPython 2.6	none	Windows x86-32	Details

Total release size: 818.1 kB

Release files / ruamel.yaml-0.9.7.tar.gz

Download URL	ruamel.yaml-0.9.7.tar.gz
Size	171.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`9a49dbaac2d87613962c8fa8d02fa7a609e9cd80e350fd7f417e25d222a9f273`
BLAKE2b-256 checksum How to use checksums	`082e36ad3bba826620494292ad7c14beb21695e6da91705f50cf228fb0fa4409`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp34-none-win_amd64.whl

Download URL	ruamel.yaml-0.9.7-cp34-none-win_amd64.whl
Size	72.3 kB
Tags	CPython 3.4 Windows x86-64
SHA-256 checksum How to use checksums	`4f9e79eb09b4979ba37fefbf2bc8d41e16782268f35d4df5da6f8cd23042c8da`
BLAKE2b-256 checksum How to use checksums	`b64483bd1fc317cf28af8417440e7749232a6b0ea515b04a258981f634f665fe`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp34-none-win32.whl

Download URL	ruamel.yaml-0.9.7-cp34-none-win32.whl
Size	72.3 kB
Tags	CPython 3.4 Windows x86-32
SHA-256 checksum How to use checksums	`d4cfa03251cd22f1406efbd02eb6a170d6f9c883ff40eeb2e1ade43af25f728f`
BLAKE2b-256 checksum How to use checksums	`7064e58f63cde2bb70bc3a613ad0612d9e116a311a096aa4bd12110a555e8fe2`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp33-none-win_amd64.whl

Download URL	ruamel.yaml-0.9.7-cp33-none-win_amd64.whl
Size	72.3 kB
Tags	CPython 3.3 Windows x86-64
SHA-256 checksum How to use checksums	`cb6701aee0738953a53ac37ef1fe9282f6b2e5279effb90a4ac7919be52d45b4`
BLAKE2b-256 checksum How to use checksums	`7eae96395436deedb325ac6cb6b1ba2349a4eb0ef2628ea766a2f7ddbacf6d4b`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp33-none-win32.whl

Download URL	ruamel.yaml-0.9.7-cp33-none-win32.whl
Size	72.3 kB
Tags	CPython 3.3 Windows x86-32
SHA-256 checksum How to use checksums	`1f284f1231435b790a8064ca9e88c598f1754d4a9daefffb29ea9aee0fd450cf`
BLAKE2b-256 checksum How to use checksums	`0f95d0bf99dc36955758ed0d5edc23b95a1943a931f1d87ba7576b7cf389af96`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp27-none-win_amd64.whl

Download URL	ruamel.yaml-0.9.7-cp27-none-win_amd64.whl
Size	72.4 kB
Tags	CPython 2.7 Windows x86-64
SHA-256 checksum How to use checksums	`9c67f4025cf4d65cfee68120da50a21a9cab0f2796906c61b2e2710cd91cea8c`
BLAKE2b-256 checksum How to use checksums	`97166036ec8ff2529a2b0d7a69611ecdf41e89bbaec179bb28069b3958db5f3d`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp27-none-win32.whl

Download URL	ruamel.yaml-0.9.7-cp27-none-win32.whl
Size	140.6 kB
Tags	CPython 2.7 Windows x86-32
SHA-256 checksum How to use checksums	`e0abef9b52e627026938ca736105e540544ce6adafe9c986ff7122de05b3cfbe`
BLAKE2b-256 checksum How to use checksums	`2b7975afdd7ac9872bc40f539814b56234f49161f23eaf5e8929ee606ccb1a38`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp26-none-win_amd64.whl

Download URL	ruamel.yaml-0.9.7-cp26-none-win_amd64.whl
Size	72.4 kB
Tags	CPython 2.6 Windows x86-64
SHA-256 checksum How to use checksums	`c3c868b4fe9c8894a2e6594d340c4b1c395784ba0fc22bd7b0b567ae33c06086`
BLAKE2b-256 checksum How to use checksums	`dc4d8cfeb9ccd212b1f0e83b6315baac40b795f1ca3721241bc002a70444d8bd`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / ruamel.yaml-0.9.7-cp26-none-win32.whl

Download URL	ruamel.yaml-0.9.7-cp26-none-win32.whl
Size	72.4 kB
Tags	CPython 2.6 Windows x86-32
SHA-256 checksum How to use checksums	`887ac75bb3290c16ca5834d1ceedde52af754df3f73046691de8257375aa89c6`
BLAKE2b-256 checksum How to use checksums	`033e60d6023da2a3a4e543d0c848e13e2f579ecf9b47c0309e1d68dc325d9fe4`
Upload date	Jun 13, 2015
Uploaded using Trusted Publishing? What is trusted publishing?	No