Skip to main content

ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order

Project description

ruamel.yaml

Starting with 0.10.7 the package has been reorganise and the commandline utility is in its own package ruamel.yaml.cmd (so installing ruamel.yaml doesn’t pull in possible irrelevant modules only used in the commandline utility)

ruamel.yaml is a YAML package for Python. It is a derivative of Kirill Simonov’s PyYAML 3.11 which supports YAML1.1

Major differences with PyYAML 3.11:

  • intergrated Python 2 and 3 sources, running on Python 2.6, 2.7 (CPython, PyPy), 3.3 and 3.4.

  • round trip mode that includes comments (block mode, key ordering kept)

  • support for simple lists as mapping keys by transformation to tuples

  • !!omap generates ordereddict (C) on Python 2, collections.OrderedDict on Python 3, and !!omap is generated for these types.

  • some YAML 1.2 enhancements (0o octal prefix, \/ escape)

  • pep8 compliance

  • tox and py.test based testing

  • Tests whether the C yaml library is installed as well as the header files. That library doesn’t generate CommentTokens, so it cannot be used to do round trip editing on comments. It can be used for speeded up normal processing (so you don’t need to install ruamel.yaml and PyYaml). See the section Optional requirements.

  • Basic support for multiline strings with preserved newlines and chomping ( ‘|’, ‘|+’, ‘|-’ ). As this subclasses the string type the information is lost on reassignment. (This might be changed in the future so that the preservation/folding/chomping is part of the parent container, like comments).

  • RoundTrip preservation of flow style sequences ( ‘a: b, c, d’) (based on request and test by Anthony Sottile)

  • anchors names that are hand-crafted (not of the form``idNNN``), are preserved

  • merges in dictionaries are preserved

  • adding/replacing of comments on block style sequences and mappings with smart column positioning

  • collection objects (when read in via RoundTripParser) have an lc property that contains line and column info lc.line and lc.col

  • preservation of whitelines after block scalars. Contributed by Sam Thursfield.

Round trip including comments

The major motivation for this fork is the round-trip capability for comments. The integration of the sources was just an initial step to make this easier.

adding/replacing comments

Starting with version 0.8, you can add/replace comments on block style collections (mappings/sequences resuting in Python dict/list). The basic for for this is:

from __future__ import print_function

import ruamel.yaml

inp = """\
abc:
  - a     # comment 1
xyz:
  a: 1    # comment 2
  b: 2
  c: 3
  d: 4
  e: 5
  f: 6 # comment 3
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
data['abc'].append('b')
data['abc'].yaml_add_eol_comment('comment 4', 1)  # takes column of comment 1
data['xyz'].yaml_add_eol_comment('comment 5', 'c')  # takes column of comment 2
data['xyz'].yaml_add_eol_comment('comment 6', 'e')  # takes column of comment 3
data['xyz'].yaml_add_eol_comment('comment 7', 'd', column=20)

print(ruamel.yaml.dump(data, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in:

abc:
- a       # comment 1
- b       # comment 4
xyz:
  a: 1    # comment 2
  b: 2
  c: 3    # comment 5
  d: 4              # comment 7
  e: 5 # comment 6
  f: 6 # comment 3

If the comment doesn’t start with ‘#’, this will be added. The key is is the element index for list, the actual key for dictionaries. As can be seen from the example, the column to choose for a comment is derived from the previous, next or preceding comment column (picking the first one found).

Config file formats

There are only a few configuration file formats that are easily readable, and editable: JSON, INI/ConfigParser, YAML (XML is to cluttered to be called easily readable).

Unfortunately JSON doesn’t support comments, and although there are some solutions with pre-processed filtering of comments, there are no libraries that support round trip updating of such commented files.

INI files support comments, and the excellent ConfigObj library by Foord and Larosa even supports round trip editing with comment preservation, nesting of sections and limited lists (within a value). Retrieval of particular value format is explicit (and extensible).

YAML has basic mapping and sequence structures as well support for ordered mappings and sets. It supports scalars are of various types including dates and datetimes (missing in JSON) as a list of YAML has comments, but these are normally thrown away.

Block structured YAML is a clean and very human readable format. By extending the Python YAML parser to support round trip preservation of comments, it makes YAML a very good choice for configuration files that are human readable and editable while at the same time interpretable and modifiable by a program.

Extending

There are normally 6 files involved when extending the roundtrip capabilities: the reader, parser, composer and constructor to go from YAML to Python and the resolver, representer, serializer and emitter to go the other way.

Extending involves keeping extra data around for the next process step, eventuallly resulting in a different Python object (subclass or alternative), that should behave like the original, but on the way from Python to YAML generates the original (or at least something much closer).

Smartening

When you use round-tripping, then the complex data you get are already subclasses of the build in types. So you can patch in extra methods or override existing ones. Some methods are already included and you can do:

yaml_str = """\
a:
- b:
  c: 42
- d:
    f: 196
  e:
    g: 3.14
"""


data = yaml.load(yaml_str, Loader=yaml.RoundTripLoader)

assert data.mlget(['a', 1, 'd', 'f'], list_ok=True) == 196

Examples

Basic round trip of parsing YAML to Python objects, modifying and generating YAML:

from __future__ import print_function

import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

code = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
code['name']['given'] = 'Bob'

print(ruamel.yaml.dump(code, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

YAML handcrafted anchors and references as well as key merging is preserved. The merged keys can transparently be accessed using [] and .get():

import ruamel.yaml

inp = """\
- &CENTER {x: 1, y: 2}
- &LEFT {x: 0, y: 2}
- &BIG {r: 10}
- &SMALL {r: 1}
# All the following maps are equal:
# Explicit keys
- x: 1
  y: 2
  r: 10
  label: center/big
# Merge one map
- <<: *CENTER
  r: 10
  label: center/big
# Merge multiple maps
- <<: [*CENTER, *BIG]
  label: center/big
# Override
- <<: [*BIG, *LEFT, *SMALL]
  x: 1
  label: center/big
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
assert data[7]['y'] == 2

Optional requirements

If you have the C yaml library and headers installed, as well as the header files for your Python executables then you can use the non-roundtrip but faster C loader en emitter.

On Debian systems you should use:

sudo apt-get install libyaml-dev python-dev python3-dev

you can leave out python3-dev if you don’t use python3

For CentOS (7) based systems you should do:

sudo yum install libyaml-devel python-devel

Testing

Testing is done using the tox, which uses virtualenv and pytest.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ruamel.yaml-0.10.9.tar.gz (233.4 kB view details)

Uploaded Source

Built Distributions

ruamel.yaml-0.10.9-cp34-none-win_amd64.whl (69.7 kB view details)

Uploaded CPython 3.4Windows x86-64

ruamel.yaml-0.10.9-cp34-none-win32.whl (69.7 kB view details)

Uploaded CPython 3.4Windows x86

ruamel.yaml-0.10.9-cp33-none-win_amd64.whl (69.7 kB view details)

Uploaded CPython 3.3Windows x86-64

ruamel.yaml-0.10.9-cp33-none-win32.whl (69.7 kB view details)

Uploaded CPython 3.3Windows x86

ruamel.yaml-0.10.9-cp27-none-win_amd64.whl (69.8 kB view details)

Uploaded CPython 2.7Windows x86-64

ruamel.yaml-0.10.9-cp27-none-win32.whl (69.8 kB view details)

Uploaded CPython 2.7Windows x86

ruamel.yaml-0.10.9-cp26-none-win_amd64.whl (69.8 kB view details)

Uploaded CPython 2.6Windows x86-64

ruamel.yaml-0.10.9-cp26-none-win32.whl (69.8 kB view details)

Uploaded CPython 2.6Windows x86

File details

Details for the file ruamel.yaml-0.10.9.tar.gz.

File metadata

  • Download URL: ruamel.yaml-0.10.9.tar.gz
  • Upload date:
  • Size: 233.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for ruamel.yaml-0.10.9.tar.gz
Algorithm Hash digest
SHA256 1ad579a599ef7ec4d8b127f01ec095f77f4704cc00887bc833353635e15a7bb7
MD5 9b91c4a320c34123658fe6cb728c7716
BLAKE2b-256 32665398b1c22329fa8a5a6c4a806d96809110393039a49199f0f16b16ba25f8

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp34-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp34-none-win_amd64.whl
Algorithm Hash digest
SHA256 1e10f6f2ad538811e29bfb071a2e7f81eee302cd03c088068f5c85ae6b5c2b1c
MD5 ed0cb5e70ed69f1e7c16d4ed5c1fb46d
BLAKE2b-256 b89b102620aec5c54a9e44dc2d75f772eb295c0f7f9f06f8cf97749adbb82617

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp34-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp34-none-win32.whl
Algorithm Hash digest
SHA256 fc1b9f6953340c4dc14ddf1c7cf874b1202d3e0daa33f1363155eb2520d2f7a7
MD5 d539fc69e0ddd04550f4b357e3798b61
BLAKE2b-256 e88c7e33c5717a488b89919bc5131d89b56e8711b4e60a40c13cafcccaf8ad34

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp33-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp33-none-win_amd64.whl
Algorithm Hash digest
SHA256 7a8468898d617b31486185f694aa95a21878b8d99648edd184aa8c8d00c3e54d
MD5 f2c4a98f46d2ad979a9da11e05c53405
BLAKE2b-256 353aa8370af2ed44af3080f1c1bac5e81c8bbe044cc577ffdc54234f80a3c0b9

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp33-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp33-none-win32.whl
Algorithm Hash digest
SHA256 9ec2378639320553f634b1c73ba8f478981d8609a73ff53db6d69e1cd46a9acb
MD5 9e83c3caad2940c44b08d5193647c8b1
BLAKE2b-256 d2ec7291fdc72796354a150844c1a3e03d5a5f4b39c6483c53c56d18ab6497aa

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp27-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp27-none-win_amd64.whl
Algorithm Hash digest
SHA256 9b6ea7985ad7a67e639e678c4cb7a26af72fca0f1d1f971fcadf8814ed73e383
MD5 592df423a6616c273fb46a9db7b9af67
BLAKE2b-256 856e359542e45c82c4a049700a3d18af1eeb7a942dfd83bda1f45d8e9cf5a089

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp27-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp27-none-win32.whl
Algorithm Hash digest
SHA256 6c9464b9b49e4fc56a9930a8a9b0a3dd27977f03ab359e25e2695e6dd29fd1f2
MD5 243db252c3dda1becc6acf990e92b65b
BLAKE2b-256 2d0fb6cb415fd80a9793bc6338b3eb281284269ed829ae1928364fce2a24f797

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp26-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp26-none-win_amd64.whl
Algorithm Hash digest
SHA256 cdd3143312e08cb55c06804ddab676f9fa9285b32788dde8bed279716f91104e
MD5 4f8a369e51cc5661c2d19e4e3d6a7bc4
BLAKE2b-256 a5acec91f22252a62ce29bba68fbe1176a2ca7414f0af639cfbf6119ebce8fd5

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.9-cp26-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.9-cp26-none-win32.whl
Algorithm Hash digest
SHA256 2f17d9c929878b328c16249a6987def67f382f8daff9184801aafca8eab68b09
MD5 4d0592f304b4a17b5fd28e8cab88ceff
BLAKE2b-256 51d35e78eee95f5f2cbc4fb7ceaf6b77663c6b9b4f1f4bf8c4584d5335895778

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page