Skip to main content

ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order

Project description

ruamel.yaml

ruamel.yaml is a YAML package for Python. It is a derivative of Kirill Simonov’s PyYAML 3.11 which supports YAML1.1

Major differences with PyYAML 3.11:

  • intergrated Python 2 and 3 sources, running on Python 2.6, 2.7 (CPython, PyPy), 3.3 and 3.4.

  • round trip mode that includes comments (block mode, key ordering kept)

  • support for simple lists as mapping keys by transformation to tuples

  • !!omap generates ordereddict (C) on Python 2, collections.OrderedDict on Python 3, and !!omap is generated for these types.

  • some YAML 1.2 enhancements (0o octal prefix, \/ escape)

  • pep8 compliance

  • tox and py.test based testing

  • Tests whether the C yaml library is installed as well as the header files. That library doesn’t generate CommentTokens, so it cannot be used to do round trip editing on comments. It can be used for speeded up normal processing (so you don’t need to install ruamel.yaml and PyYaml). See the section Optional requirements.

  • Basic support for multiline strings with preserved newlines and chomping ( ‘|’, ‘|+’, ‘|-’ ). As this subclasses the string type the information is lost on reassignment. (This might be changed in the future so that the preservation/folding/chomping is part of the parent container, like comments).

  • RoundTrip preservation of flow style sequences ( ‘a: b, c, d’) (based on request and test by Anthony Sottile)

  • adding/replacing of comments on block style sequences and mappings with smart column positioning

  • collection objects (when read in via RoundTripParser) have an lc property that contains line and column info lc.line and lc.col

Round trip including comments

The major motivation for this fork is the round-trip capability for comments. The integration of the sources was just an initial step to make this easier.

adding/replacing comments

Starting with version 0.8, you can add/replace comments on block style collections (mappings/sequences resuting in Python dict/list). The basic for for this is:

from __future__ import print_function

import ruamel.yaml

inp = """\
abc:
  - a     # comment 1
xyz:
  a: 1    # comment 2
  b: 2
  c: 3
  d: 4
  e: 5
  f: 6 # comment 3
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
data['abc'].append('b')
data['abc'].yaml_add_eol_comment('comment 4', 1)  # takes column of comment 1
data['xyz'].yaml_add_eol_comment('comment 5', 'c')  # takes column of comment 2
data['xyz'].yaml_add_eol_comment('comment 6', 'e')  # takes column of comment 3
data['xyz'].yaml_add_eol_comment('comment 7', 'd', column=20)

print(ruamel.yaml.dump(data, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in:

abc:
- a       # comment 1
- b       # comment 4
xyz:
  a: 1    # comment 2
  b: 2
  c: 3    # comment 5
  d: 4              # comment 7
  e: 5 # comment 6
  f: 6 # comment 3

If the comment doesn’t start with ‘#’, this will be added. The key is is the element index for list, the actual key for dictionaries. As can be seen from the example, the column to choose for a comment is derived from the previous, next or preceding comment column (picking the first one found).

Config file formats

There are only a few configuration file formats that are easily readable, and editable: JSON, INI/ConfigParser, YAML (XML is to cluttered to be called easily readable).

Unfortunately JSON doesn’t support comments, and although there are some solutions with pre-processed filtering of comments, there are no libraries that support round trip updating of such commented files.

INI files support comments, and the excellent ConfigObj library by Foord and Larosa even supports round trip editing with comment preservation, nesting of sections and limited lists (within a value). Retrieval of particular value format is explicit (and extensible).

YAML has basic mapping and sequence structures as well support for ordered mappings and sets. It supports scalars are of various types including dates and datetimes (missing in JSON) as a list of YAML has comments, but these are normally thrown away.

Block structured YAML is a clean and very human readable format. By extending the Python YAML parser to support round trip preservation of comments, it makes YAML a very good choice for configuration files that are human readable and editable while at the same time interpretable and modifiable by a program.

Extending

There are normally 6 files involved when extending the roundtrip capabilities: the reader, parser, composer and constructor to go from YAML to Python and the resolver, representer, serializer and emitter to go the other way.

Extending involves keeping extra data around for the next process step, eventuallly resulting in a different Python object (subclass or alternative), that should behave like the original, but on the way from Python to YAML generates the original (or at least something much closer).

Smartening

When you use round-tripping, then the complex data you get are already subclasses of the build in types. So you can patch in extra methods or override existing ones. Some methods are already included and you can do:

yaml_str = """\
a:
- b:
  c: 42
- d:
    f: 196
  e:
    g: 3.14
"""


data = yaml.load(yaml_str, Loader=yaml.RoundTripLoader)

assert data.mlget(['a', 1, 'd', 'f'], list_ok=True) == 196

Examples

Basic round trip of parsing YAML to Python objects, modifying and generating YAML:

from __future__ import print_function

import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

code = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
code['name']['given'] = 'Bob'

print(ruamel.yaml.dump(code, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

Optional requirements

If you have the C yaml library and headers installed, as well as the header files for your Python executables then you can use the non-roundtrip but faster C loader en emitter.

On Debian systems you should use:

sudo apt-get install libyaml-dev python-dev python3-dev

you can leave out python3-dev if you don’t use python3

For CentOS (7) based systems you should do:

sudo yum install libyaml-devel python-devel

Testing

Testing is done using the tox, which uses virtualenv and pytest.

yaml utlity

A utility name yaml is included and allows for basic operations on files:

  • yaml round-trip <file_name> for basic roundtrip testing of YAML files

  • yaml json <file_name> for conversion of JSON file(s) to a single YAML block style document

  • yaml ini <file_name> for conversion of an INI/config file (ConfigObj comment and nested sections supported) to a YAML block style document. This requires configobj to be installed (pip install configobj)

  • yaml html <file_name> for conversion of the basic structure in a YAML file to a a table in an HTML file. The YAML file:

    title:
    - fruit
    - legume
    local:
    - apple
    - sprouts
    import:
    - orange
    - broccoli

    is converted into the table:

    title

    fruit

    legume

    local

    apple

    sprouts

    import

    orange

    broccoli

See yaml --help for more information on the availble commands

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ruamel.yaml-0.9.7.tar.gz (171.0 kB view details)

Uploaded Source

Built Distributions

ruamel.yaml-0.9.7-cp34-none-win_amd64.whl (72.3 kB view details)

Uploaded CPython 3.4Windows x86-64

ruamel.yaml-0.9.7-cp34-none-win32.whl (72.3 kB view details)

Uploaded CPython 3.4Windows x86

ruamel.yaml-0.9.7-cp33-none-win_amd64.whl (72.3 kB view details)

Uploaded CPython 3.3Windows x86-64

ruamel.yaml-0.9.7-cp33-none-win32.whl (72.3 kB view details)

Uploaded CPython 3.3Windows x86

ruamel.yaml-0.9.7-cp27-none-win_amd64.whl (72.4 kB view details)

Uploaded CPython 2.7Windows x86-64

ruamel.yaml-0.9.7-cp27-none-win32.whl (140.6 kB view details)

Uploaded CPython 2.7Windows x86

ruamel.yaml-0.9.7-cp26-none-win_amd64.whl (72.4 kB view details)

Uploaded CPython 2.6Windows x86-64

ruamel.yaml-0.9.7-cp26-none-win32.whl (72.4 kB view details)

Uploaded CPython 2.6Windows x86

File details

Details for the file ruamel.yaml-0.9.7.tar.gz.

File metadata

  • Download URL: ruamel.yaml-0.9.7.tar.gz
  • Upload date:
  • Size: 171.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for ruamel.yaml-0.9.7.tar.gz
Algorithm Hash digest
SHA256 9a49dbaac2d87613962c8fa8d02fa7a609e9cd80e350fd7f417e25d222a9f273
MD5 bb804329364da017122e1ff499e4949e
BLAKE2b-256 082e36ad3bba826620494292ad7c14beb21695e6da91705f50cf228fb0fa4409

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp34-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp34-none-win_amd64.whl
Algorithm Hash digest
SHA256 4f9e79eb09b4979ba37fefbf2bc8d41e16782268f35d4df5da6f8cd23042c8da
MD5 ca72b619bc254d3f28d0eb620f52a7ca
BLAKE2b-256 b64483bd1fc317cf28af8417440e7749232a6b0ea515b04a258981f634f665fe

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp34-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp34-none-win32.whl
Algorithm Hash digest
SHA256 d4cfa03251cd22f1406efbd02eb6a170d6f9c883ff40eeb2e1ade43af25f728f
MD5 f3ec47b45ebb8cf9652a7ae9ad41e1d6
BLAKE2b-256 7064e58f63cde2bb70bc3a613ad0612d9e116a311a096aa4bd12110a555e8fe2

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp33-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp33-none-win_amd64.whl
Algorithm Hash digest
SHA256 cb6701aee0738953a53ac37ef1fe9282f6b2e5279effb90a4ac7919be52d45b4
MD5 e04c135539b66f79ca674c7a3e5522e0
BLAKE2b-256 7eae96395436deedb325ac6cb6b1ba2349a4eb0ef2628ea766a2f7ddbacf6d4b

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp33-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp33-none-win32.whl
Algorithm Hash digest
SHA256 1f284f1231435b790a8064ca9e88c598f1754d4a9daefffb29ea9aee0fd450cf
MD5 64878b8113c60f8e41b105af5c0bb2b2
BLAKE2b-256 0f95d0bf99dc36955758ed0d5edc23b95a1943a931f1d87ba7576b7cf389af96

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp27-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp27-none-win_amd64.whl
Algorithm Hash digest
SHA256 9c67f4025cf4d65cfee68120da50a21a9cab0f2796906c61b2e2710cd91cea8c
MD5 73ec6da95fced109bbd6ae70fdf69cbf
BLAKE2b-256 97166036ec8ff2529a2b0d7a69611ecdf41e89bbaec179bb28069b3958db5f3d

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp27-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp27-none-win32.whl
Algorithm Hash digest
SHA256 e0abef9b52e627026938ca736105e540544ce6adafe9c986ff7122de05b3cfbe
MD5 b83703c50d39da75816c7892719ca00d
BLAKE2b-256 2b7975afdd7ac9872bc40f539814b56234f49161f23eaf5e8929ee606ccb1a38

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp26-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp26-none-win_amd64.whl
Algorithm Hash digest
SHA256 c3c868b4fe9c8894a2e6594d340c4b1c395784ba0fc22bd7b0b567ae33c06086
MD5 97668e5f0cd7dfa5371c877894115f02
BLAKE2b-256 dc4d8cfeb9ccd212b1f0e83b6315baac40b795f1ca3721241bc002a70444d8bd

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.7-cp26-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.7-cp26-none-win32.whl
Algorithm Hash digest
SHA256 887ac75bb3290c16ca5834d1ceedde52af754df3f73046691de8257375aa89c6
MD5 569927e4bfe57fc69f586260d7f65183
BLAKE2b-256 033e60d6023da2a3a4e543d0c848e13e2f579ecf9b47c0309e1d68dc325d9fe4

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page