Skip to main content

ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order

Project description

ruamel.yaml

ruamel.yaml is a YAML package for Python. It is a derivative of Kirill Simonov’s PyYAML 3.11 which supports YAML1.1

Major differences with PyYAML 3.11:

  • intergrated Python 2 and 3 sources, running on Python 2.6, 2.7 (CPython, PyPy), 3.3 and 3.4.

  • round trip mode that includes comments (block mode, key ordering kept)

  • support for simple lists as mapping keys by transformation to tuples

  • !!omap generates ordereddict (C) on Python 2, collections.OrderedDict on Python 3, and !!omap is generated for these types.

  • some YAML 1.2 enhancements (0o octal prefix, \/ escape)

  • pep8 compliance

  • tox and py.test based testing

  • Tests whether the C yaml library is installed as well as the header files. That library doesn’t generate CommentTokens, so it cannot be used to do round trip editing on comments. It can be used for speeded up normal processing (so you don’t need to install ruamel.yaml and PyYaml). See the section Optional requirements.

  • Basic support for multiline strings with preserved newlines and chomping ( ‘|’, ‘|+’, ‘|-’ ). As this subclasses the string type the information is lost on reassignment. (This might be changed in the future so that the preservation/folding/chomping is part of the parent container, like comments).

  • RoundTrip preservation of flow style sequences ( ‘a: b, c, d’) (based on request and test by Anthony Sottile)

  • adding/replacing of comments on block style sequences and mappings with smart column positioning

  • collection objects (when read in via RoundTripParser) have an lc property that contains line and column info lc.line and lc.col

Round trip including comments

The major motivation for this fork is the round-trip capability for comments. The integration of the sources was just an initial step to make this easier.

adding/replacing comments

Starting with version 0.8, you can add/replace comments on block style collections (mappings/sequences resuting in Python dict/list). The basic for for this is:

from __future__ import print_function

import ruamel.yaml

inp = """\
abc:
  - a     # comment 1
xyz:
  a: 1    # comment 2
  b: 2
  c: 3
  d: 4
  e: 5
  f: 6 # comment 3
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
data['abc'].append('b')
data['abc'].yaml_add_eol_comment('comment 4', 1)  # takes column of comment 1
data['xyz'].yaml_add_eol_comment('comment 5', 'c')  # takes column of comment 2
data['xyz'].yaml_add_eol_comment('comment 6', 'e')  # takes column of comment 3
data['xyz'].yaml_add_eol_comment('comment 7', 'd', column=20)

print(ruamel.yaml.dump(data, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in:

abc:
- a       # comment 1
- b       # comment 4
xyz:
  a: 1    # comment 2
  b: 2
  c: 3    # comment 5
  d: 4              # comment 7
  e: 5 # comment 6
  f: 6 # comment 3

If the comment doesn’t start with ‘#’, this will be added. The key is is the element index for list, the actual key for dictionaries. As can be seen from the example, the column to choose for a comment is derived from the previous, next or preceding comment column (picking the first one found).

Config file formats

There are only a few configuration file formats that are easily readable, and editable: JSON, INI/ConfigParser, YAML (XML is to cluttered to be called easily readable).

Unfortunately JSON doesn’t support comments, and although there are some solutions with pre-processed filtering of comments, there are no libraries that support round trip updating of such commented files.

INI files support comments, and the excellent ConfigObj library by Foord and Larosa even supports round trip editing with comment preservation, nesting of sections and limited lists (within a value). Retrieval of particular value format is explicit (and extensible).

YAML has basic mapping and sequence structures as well support for ordered mappings and sets. It supports scalars are of various types including dates and datetimes (missing in JSON) as a list of YAML has comments, but these are normally thrown away.

Block structured YAML is a clean and very human readable format. By extending the Python YAML parser to support round trip preservation of comments, it makes YAML a very good choice for configuration files that are human readable and editable while at the same time interpretable and modifiable by a program.

Extending

There are normally 6 files involved when extending the roundtrip capabilities: the reader, parser, composer and constructor to go from YAML to Python and the resolver, representer, serializer and emitter to go the other way.

Extending involves keeping extra data around for the next process step, eventuallly resulting in a different Python object (subclass or alternative), that should behave like the original, but on the way from Python to YAML generates the original (or at least something much closer).

Examples

Basic round trip of parsing YAML to Python objects, modifying and generating YAML:

from __future__ import print_function

import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

code = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
code['name']['given'] = 'Bob'

print(ruamel.yaml.dump(code, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

Optional requirements

If you have the C yaml library and headers installed, as well as the header files for your Python executables then you can use the non-roundtrip but faster C loader en emitter.

On Debian systems you should use:

sudo apt-get install libyaml-dev python-dev python3-dev

you can leave out python3-dev if you don’t use python3

For CentOS (7) based systems you should do:

sudo yum install libyaml-devel python-devel

Testing

Testing is done using the tox, which uses virtualenv and pytest.

yaml utlity

A utility name yaml is included and allows for basic operations on files:

  • yaml round-trip <file_name> for basic roundtrip testing of YAML files

  • yaml json <file_name> for conversion of JSON file(s) to a single YAML block style document

  • yaml ini <file_name> for conversion of an INI/config file (ConfigObj comment and nested sections supported) to a YAML block style document. This requires configobj to be installed (pip install configobj)

  • yaml html <file_name> for conversion of the basic structure in a YAML file to a a table in an HTML file. The YAML file:

    title:
    - fruit
    - legume
    local:
    - apple
    - sprouts
    import:
    - orange
    - broccoli

    is converted into the table:

    title

    fruit

    legume

    local

    apple

    sprouts

    import

    orange

    broccoli

See yaml --help for more information on the availble commands

Project details


Release history Release notifications | RSS feed

This version

0.9.6

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ruamel.yaml-0.9.6.tar.gz (170.1 kB view details)

Uploaded Source

Built Distributions

ruamel.yaml-0.9.6-cp34-none-win_amd64.whl (71.5 kB view details)

Uploaded CPython 3.4Windows x86-64

ruamel.yaml-0.9.6-cp34-none-win32.whl (71.5 kB view details)

Uploaded CPython 3.4Windows x86

ruamel.yaml-0.9.6-cp33-none-win_amd64.whl (71.5 kB view details)

Uploaded CPython 3.3Windows x86-64

ruamel.yaml-0.9.6-cp33-none-win32.whl (71.5 kB view details)

Uploaded CPython 3.3Windows x86

ruamel.yaml-0.9.6-cp27-none-win_amd64.whl (71.6 kB view details)

Uploaded CPython 2.7Windows x86-64

ruamel.yaml-0.9.6-cp27-none-win32.whl (71.6 kB view details)

Uploaded CPython 2.7Windows x86

ruamel.yaml-0.9.6-cp26-none-win_amd64.whl (71.6 kB view details)

Uploaded CPython 2.6Windows x86-64

ruamel.yaml-0.9.6-cp26-none-win32.whl (71.6 kB view details)

Uploaded CPython 2.6Windows x86

File details

Details for the file ruamel.yaml-0.9.6.tar.gz.

File metadata

  • Download URL: ruamel.yaml-0.9.6.tar.gz
  • Upload date:
  • Size: 170.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for ruamel.yaml-0.9.6.tar.gz
Algorithm Hash digest
SHA256 9730ba3f094e4f49f279aa0d0d2caf0b99cd80f4fff9d04f565597d678aa9163
MD5 0f5876cc5f9be4bb696499666b45de2a
BLAKE2b-256 5ed2c0c2cbbdfb089b61ac1562522fdb7e5bf2bc8cc69b9a666a3b49eb27b7d8

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp34-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp34-none-win_amd64.whl
Algorithm Hash digest
SHA256 c45dde7084de690dcdeff8934756433da33e7fd2bf0da30354fff86c3792b3a5
MD5 f00361d2489ab1f92e7431372a89fa29
BLAKE2b-256 05bbe4918bf7e3f4f0dddc8657fa882dd7d74dbcb9d69d62a65cf24797327adc

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp34-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp34-none-win32.whl
Algorithm Hash digest
SHA256 d70c1d3f50e85c9679e69287f75fc91a84f241bf9e912cd5d769f1966b942a6b
MD5 8aa5894f57480b4639ac7281ecb9780e
BLAKE2b-256 91e9743172ece80b063c0d743408586754b4ec95b3a42479e65c4d041a0366aa

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp33-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp33-none-win_amd64.whl
Algorithm Hash digest
SHA256 2c438f971b360bdf9395ea4a4209d537704b4c23190c9cd0da00850f059736ef
MD5 e759e970c6d21c9c596c94b6cd5f4bfb
BLAKE2b-256 ab228988cff2a00db2de9203077a1b427223223ecf44f8b0dedfc85f81170be8

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp33-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp33-none-win32.whl
Algorithm Hash digest
SHA256 934c19cc35aab29fc8d4fe120c7453f02a9b35921a52bd24e60a132856d70d7e
MD5 2447e8603e1b32e76b41440a15cea420
BLAKE2b-256 edc00e38db4eebd09227a4ec2e37b30ce6f0257b5a4813766d516df273442faf

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp27-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp27-none-win_amd64.whl
Algorithm Hash digest
SHA256 a43659b6bf97fad254f5977f9ab62166d93ccf2cfde99c319d8b7a4f520b4852
MD5 19c26b85b90930561b278ac776bd8e6d
BLAKE2b-256 bc9425a0df4b4616ba7b74957019ec6937483776d1a274197fd3cb4f3d57531a

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp27-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp27-none-win32.whl
Algorithm Hash digest
SHA256 2e7edd2b0549b8bfe81fa9b24136b5b0026de39b2c7c7ea9742b120ba4413ace
MD5 d240d574e48f2a8af2c3c674884fe065
BLAKE2b-256 9976cbb922f124b97d132fef3372ce93c211480e8acabc248723b0d955c5aa6f

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp26-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp26-none-win_amd64.whl
Algorithm Hash digest
SHA256 a69ab369c2a423c554967ac8d709ead02703553d26cb9fdf2f0045cb545a66a5
MD5 8a615819d2f7292043a4222326b8e8e1
BLAKE2b-256 43a8e01bbdc66e3c3cc4c383ea276737ad1cb37542f1415845328822bb3729ee

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.9.6-cp26-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.9.6-cp26-none-win32.whl
Algorithm Hash digest
SHA256 fa659e9ced6c088d965733ecf9aa9dfd40f8adbaaf56ef47f64bd414c0f48f12
MD5 9338c2881d3b1f541ea98cd9f39e8339
BLAKE2b-256 57d655dfe322ee969c13507957539ed1a106ebaf4fde868465f7add996ce9d57

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page