Skip to main content

ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order

Project description

ruamel.yaml

Starting with 0.10.7 the package has been reorganise and the commandline utility is in its own package ruamel.yaml.cmd (so installing ruamel.yaml doesn’t pull in possible irrelevant modules only used in the commandline utility)

ruamel.yaml is a YAML package for Python. It is a derivative of Kirill Simonov’s PyYAML 3.11 which supports YAML1.1

Major differences with PyYAML 3.11:

  • intergrated Python 2 and 3 sources, running on Python 2.6, 2.7 (CPython, PyPy), 3.3 and 3.4.

  • round trip mode that includes comments (block mode, key ordering kept)

  • support for simple lists as mapping keys by transformation to tuples

  • !!omap generates ordereddict (C) on Python 2, collections.OrderedDict on Python 3, and !!omap is generated for these types.

  • some YAML 1.2 enhancements (0o octal prefix, \/ escape)

  • pep8 compliance

  • tox and py.test based testing

  • Tests whether the C yaml library is installed as well as the header files. That library doesn’t generate CommentTokens, so it cannot be used to do round trip editing on comments. It can be used for speeded up normal processing (so you don’t need to install ruamel.yaml and PyYaml). See the section Optional requirements.

  • Basic support for multiline strings with preserved newlines and chomping ( ‘|’, ‘|+’, ‘|-’ ). As this subclasses the string type the information is lost on reassignment. (This might be changed in the future so that the preservation/folding/chomping is part of the parent container, like comments).

  • RoundTrip preservation of flow style sequences ( ‘a: b, c, d’) (based on request and test by Anthony Sottile)

  • anchors names that are hand-crafted (not of the form``idNNN``), are preserved

  • merges in dictionaries are preserved

  • adding/replacing of comments on block style sequences and mappings with smart column positioning

  • collection objects (when read in via RoundTripParser) have an lc property that contains line and column info lc.line and lc.col. Individual positions for mappings and sequences can also be retrieved (lc.key('a'), lc.value('a') resp. lc.item(3))

  • preservation of whitelines after block scalars. Contributed by Sam Thursfield.

Round trip including comments

The major motivation for this fork is the round-trip capability for comments. The integration of the sources was just an initial step to make this easier.

adding/replacing comments

Starting with version 0.8, you can add/replace comments on block style collections (mappings/sequences resuting in Python dict/list). The basic for for this is:

from __future__ import print_function

import ruamel.yaml

inp = """\
abc:
  - a     # comment 1
xyz:
  a: 1    # comment 2
  b: 2
  c: 3
  d: 4
  e: 5
  f: 6 # comment 3
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
data['abc'].append('b')
data['abc'].yaml_add_eol_comment('comment 4', 1)  # takes column of comment 1
data['xyz'].yaml_add_eol_comment('comment 5', 'c')  # takes column of comment 2
data['xyz'].yaml_add_eol_comment('comment 6', 'e')  # takes column of comment 3
data['xyz'].yaml_add_eol_comment('comment 7', 'd', column=20)

print(ruamel.yaml.dump(data, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in:

abc:
- a       # comment 1
- b       # comment 4
xyz:
  a: 1    # comment 2
  b: 2
  c: 3    # comment 5
  d: 4              # comment 7
  e: 5 # comment 6
  f: 6 # comment 3

If the comment doesn’t start with ‘#’, this will be added. The key is is the element index for list, the actual key for dictionaries. As can be seen from the example, the column to choose for a comment is derived from the previous, next or preceding comment column (picking the first one found).

Config file formats

There are only a few configuration file formats that are easily readable, and editable: JSON, INI/ConfigParser, YAML (XML is to cluttered to be called easily readable).

Unfortunately JSON doesn’t support comments, and although there are some solutions with pre-processed filtering of comments, there are no libraries that support round trip updating of such commented files.

INI files support comments, and the excellent ConfigObj library by Foord and Larosa even supports round trip editing with comment preservation, nesting of sections and limited lists (within a value). Retrieval of particular value format is explicit (and extensible).

YAML has basic mapping and sequence structures as well support for ordered mappings and sets. It supports scalars are of various types including dates and datetimes (missing in JSON) as a list of YAML has comments, but these are normally thrown away.

Block structured YAML is a clean and very human readable format. By extending the Python YAML parser to support round trip preservation of comments, it makes YAML a very good choice for configuration files that are human readable and editable while at the same time interpretable and modifiable by a program.

Extending

There are normally 6 files involved when extending the roundtrip capabilities: the reader, parser, composer and constructor to go from YAML to Python and the resolver, representer, serializer and emitter to go the other way.

Extending involves keeping extra data around for the next process step, eventuallly resulting in a different Python object (subclass or alternative), that should behave like the original, but on the way from Python to YAML generates the original (or at least something much closer).

Smartening

When you use round-tripping, then the complex data you get are already subclasses of the build in types. So you can patch in extra methods or override existing ones. Some methods are already included and you can do:

yaml_str = """\
a:
- b:
  c: 42
- d:
    f: 196
  e:
    g: 3.14
"""


data = yaml.load(yaml_str, Loader=yaml.RoundTripLoader)

assert data.mlget(['a', 1, 'd', 'f'], list_ok=True) == 196

Examples

Basic round trip of parsing YAML to Python objects, modifying and generating YAML:

from __future__ import print_function

import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

code = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
code['name']['given'] = 'Bob'

print(ruamel.yaml.dump(code, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

YAML handcrafted anchors and references as well as key merging is preserved. The merged keys can transparently be accessed using [] and .get():

import ruamel.yaml

inp = """\
- &CENTER {x: 1, y: 2}
- &LEFT {x: 0, y: 2}
- &BIG {r: 10}
- &SMALL {r: 1}
# All the following maps are equal:
# Explicit keys
- x: 1
  y: 2
  r: 10
  label: center/big
# Merge one map
- <<: *CENTER
  r: 10
  label: center/big
# Merge multiple maps
- <<: [*CENTER, *BIG]
  label: center/big
# Override
- <<: [*BIG, *LEFT, *SMALL]
  x: 1
  label: center/big
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
assert data[7]['y'] == 2

Optional requirements

If you have the C yaml library and headers installed, as well as the header files for your Python executables then you can use the non-roundtrip but faster C loader en emitter.

On Debian systems you should use:

sudo apt-get install libyaml-dev python-dev python3-dev

you can leave out python3-dev if you don’t use python3

For CentOS (7) based systems you should do:

sudo yum install libyaml-devel python-devel

Testing

Testing is done using the tox, which uses virtualenv and pytest.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ruamel.yaml-0.10.11.tar.gz (234.1 kB view details)

Uploaded Source

Built Distributions

ruamel.yaml-0.10.11-cp34-none-win_amd64.whl (70.1 kB view details)

Uploaded CPython 3.4Windows x86-64

ruamel.yaml-0.10.11-cp34-none-win32.whl (70.1 kB view details)

Uploaded CPython 3.4Windows x86

ruamel.yaml-0.10.11-cp33-none-win_amd64.whl (70.1 kB view details)

Uploaded CPython 3.3Windows x86-64

ruamel.yaml-0.10.11-cp33-none-win32.whl (70.1 kB view details)

Uploaded CPython 3.3Windows x86

ruamel.yaml-0.10.11-cp27-none-win_amd64.whl (70.2 kB view details)

Uploaded CPython 2.7Windows x86-64

ruamel.yaml-0.10.11-cp27-none-win32.whl (70.2 kB view details)

Uploaded CPython 2.7Windows x86

ruamel.yaml-0.10.11-cp26-none-win_amd64.whl (70.2 kB view details)

Uploaded CPython 2.6Windows x86-64

ruamel.yaml-0.10.11-cp26-none-win32.whl (70.2 kB view details)

Uploaded CPython 2.6Windows x86

File details

Details for the file ruamel.yaml-0.10.11.tar.gz.

File metadata

  • Download URL: ruamel.yaml-0.10.11.tar.gz
  • Upload date:
  • Size: 234.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for ruamel.yaml-0.10.11.tar.gz
Algorithm Hash digest
SHA256 da642f5e8c932db3e3a1ba54a1b6f7f20d1ddaac01936fdc200ae74cb1d51791
MD5 4f1f29a85601f9b179298a490167cd4f
BLAKE2b-256 3ee3001e22a471e65f7d14e08342a15dea8e45f55bc6ce84f26354199da9df3e

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp34-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp34-none-win_amd64.whl
Algorithm Hash digest
SHA256 d29eb8132f8011f0b279bf248a6a51261ca2ad601579faa849680b0b7d29153f
MD5 5e33d029c758c2798548d6069c1d574e
BLAKE2b-256 2117d14d542850d7001c013100c3311c4e4f569a24cde36ca0efc261c05b7f88

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp34-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp34-none-win32.whl
Algorithm Hash digest
SHA256 da2278c945d2fe68ac56301cb4d0b4fcca3d178fce29003894cabd49436032e1
MD5 85906974432a225ccba3d4b606585e77
BLAKE2b-256 eb730349ef5d4a3e69b112cb494f3a9bf1d0c01a813269271e123a4fa51b1d30

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp33-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp33-none-win_amd64.whl
Algorithm Hash digest
SHA256 5b723f79b8042d73b86f94be8560477b868d1bdcc90a780b3744412d01817242
MD5 51fb8cfbabd8810f98a330b268b93b3b
BLAKE2b-256 61e624ef1bbe6eda886d43234183f783fd83001dba5d9d89583ee172ce3bdd41

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp33-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp33-none-win32.whl
Algorithm Hash digest
SHA256 d27f8e9db46d10c40c3ee939d0ec6e316067e7516fde78bf55fd772e15efa683
MD5 dc442ae1dc7b89edf6f7dcac2ce85a3f
BLAKE2b-256 3b4f0bc14e2107ba696c220818f9e66b24c7e9a6823b2ae596bb3472359ed790

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp27-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp27-none-win_amd64.whl
Algorithm Hash digest
SHA256 46b1cf966c87fca4c67d5c441ff5c9bd28510e551097d6ae083ef04b6e160dc0
MD5 e6874154aadc3042c6defadb96aa32cb
BLAKE2b-256 5699ae6bab82ccb5de6057c924f24a253e4918788eb91a1d1c918060c1bfa96b

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp27-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp27-none-win32.whl
Algorithm Hash digest
SHA256 f707596a9a83545c93f6930927dc6d6305842af8ad17d0eaaa79b213d78c9773
MD5 cab61307860b83c85e027f03bb04cd9d
BLAKE2b-256 c5772ef8dd2f08c21d8e8af59a249c61697ffb9c2957d2bba5b544fdf8899e81

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp26-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp26-none-win_amd64.whl
Algorithm Hash digest
SHA256 0f0a826ecc1d93282baeacf3c02627016932d5bd27e4af93dc328f665266d46c
MD5 3d223600ec954e99cb4745404e35b360
BLAKE2b-256 d94caaf4bfeabab9e0943a7fbfdc35fd5bcb7e104228cfc6f35dabb0263fe805

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.11-cp26-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.11-cp26-none-win32.whl
Algorithm Hash digest
SHA256 5e32a36eaf1f9a3ce620209fe82bb8d78ac687ae015bbb7085b8e51874cb789b
MD5 8cd4c67823f059735249292566e1ed03
BLAKE2b-256 7c06211e2bbcbf3c438daae5ab2400716257ace0ad5a4614638c20f244b7bff8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page