Skip to main content

ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order

Project description

ruamel.yaml

Starting with 0.10.7 the package has been reorganised and the command line utility is in its own package ruamel.yaml.cmd (so installing ruamel.yaml doesn’t pull in possibly irrelevant modules only used in the command line utility)

ruamel.yaml is a YAML package for Python. It is a derivative of Kirill Simonov’s PyYAML 3.11 which supports YAML1.1

Major differences with PyYAML 3.11:

  • integrated Python 2 and 3 sources, running on Python 2.6, 2.7 (CPython, PyPy), 3.3 and 3.4.

  • round trip mode that includes comments (block mode, key ordering kept)

  • support for simple lists as mapping keys by transforming these to tuples

  • !!omap generates ordereddict (C) on Python 2, collections.OrderedDict on Python 3, and !!omap is generated for these types.

  • some YAML 1.2 enhancements (0o octal prefix, \/ escape)

  • pep8 compliance

  • tox and py.test based testing

  • Tests whether the C yaml library is installed as well as the header files. That library doesn’t generate CommentTokens, so it cannot be used to do round trip editing on comments. It can be used to speed up normal processing (so you don’t need to install ruamel.yaml and PyYaml). See the section Optional requirements.

  • Basic support for multiline strings with preserved newlines and chomping ( ‘|’, ‘|+’, ‘|-’ ). As this subclasses the string type the information is lost on reassignment. (This might be changed in the future so that the preservation/folding/chomping is part of the parent container, like comments).

  • RoundTrip preservation of flow style sequences ( ‘a: b, c, d’) (based on request and test by Anthony Sottile)

  • anchors names that are hand-crafted (not of the form``idNNN``) are preserved

  • merges in dictionaries are preserved

  • adding/replacing comments on block-style sequences and mappings with smart column positioning

  • collection objects (when read in via RoundTripParser) have an lc property that contains line and column info lc.line and lc.col. Individual positions for mappings and sequences can also be retrieved (lc.key('a'), lc.value('a') resp. lc.item(3))

  • preservation of whitelines after block scalars. Contributed by Sam Thursfield.

Round trip including comments

The major motivation for this fork is the round-trip capability for comments. The integration of the sources was just an initial step to make this easier.

adding/replacing comments

Starting with version 0.8, you can add/replace comments on block style collections (mappings/sequences resuting in Python dict/list). The basic for for this is:

from __future__ import print_function

import ruamel.yaml

inp = """\
abc:
  - a     # comment 1
xyz:
  a: 1    # comment 2
  b: 2
  c: 3
  d: 4
  e: 5
  f: 6 # comment 3
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
data['abc'].append('b')
data['abc'].yaml_add_eol_comment('comment 4', 1)  # takes column of comment 1
data['xyz'].yaml_add_eol_comment('comment 5', 'c')  # takes column of comment 2
data['xyz'].yaml_add_eol_comment('comment 6', 'e')  # takes column of comment 3
data['xyz'].yaml_add_eol_comment('comment 7', 'd', column=20)

print(ruamel.yaml.dump(data, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in:

abc:
- a       # comment 1
- b       # comment 4
xyz:
  a: 1    # comment 2
  b: 2
  c: 3    # comment 5
  d: 4              # comment 7
  e: 5 # comment 6
  f: 6 # comment 3

If the comment doesn’t start with ‘#’, this will be added. The key is the element index for list, the actual key for dictionaries. As can be seen from the example, the column to choose for a comment is derived from the previous, next or preceding comment column (picking the first one found).

Config file formats

There are only a few configuration file formats that are easily readable and editable: JSON, INI/ConfigParser, YAML (XML is to cluttered to be called easily readable).

Unfortunately JSON doesn’t support comments, and although there are some solutions with pre-processed filtering of comments, there are no libraries that support round trip updating of such commented files.

INI files support comments, and the excellent ConfigObj library by Foord and Larosa even supports round trip editing with comment preservation, nesting of sections and limited lists (within a value). Retrieval of particular value format is explicit (and extensible).

YAML has basic mapping and sequence structures as well as support for ordered mappings and sets. It supports scalars various types including dates and datetimes (missing in JSON). YAML has comments, but these are normally thrown away.

Block structured YAML is a clean and very human readable format. By extending the Python YAML parser to support round trip preservation of comments, it makes YAML a very good choice for configuration files that are human readable and editable while at the same time interpretable and modifiable by a program.

Extending

There are normally six files involved when extending the roundtrip capabilities: the reader, parser, composer and constructor to go from YAML to Python and the resolver, representer, serializer and emitter to go the other way.

Extending involves keeping extra data around for the next process step, eventuallly resulting in a different Python object (subclass or alternative), that should behave like the original, but on the way from Python to YAML generates the original (or at least something much closer).

Smartening

When you use round-tripping, then the complex data you get are already subclasses of the built-in types. So you can patch in extra methods or override existing ones. Some methods are already included and you can do:

yaml_str = """\
a:
- b:
  c: 42
- d:
    f: 196
  e:
    g: 3.14
"""


data = yaml.load(yaml_str, Loader=yaml.RoundTripLoader)

assert data.mlget(['a', 1, 'd', 'f'], list_ok=True) == 196

Examples

Basic round trip of parsing YAML to Python objects, modifying and generating YAML:

from __future__ import print_function

import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

code = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
code['name']['given'] = 'Bob'

print(ruamel.yaml.dump(code, Dumper=ruamel.yaml.RoundTripDumper), end='')

Resulting in

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

YAML handcrafted anchors and references as well as key merging is preserved. The merged keys can transparently be accessed using [] and .get():

import ruamel.yaml

inp = """\
- &CENTER {x: 1, y: 2}
- &LEFT {x: 0, y: 2}
- &BIG {r: 10}
- &SMALL {r: 1}
# All the following maps are equal:
# Explicit keys
- x: 1
  y: 2
  r: 10
  label: center/big
# Merge one map
- <<: *CENTER
  r: 10
  label: center/big
# Merge multiple maps
- <<: [*CENTER, *BIG]
  label: center/big
# Override
- <<: [*BIG, *LEFT, *SMALL]
  x: 1
  label: center/big
"""

data = ruamel.yaml.load(inp, ruamel.yaml.RoundTripLoader)
assert data[7]['y'] == 2

Optional requirements

If you have the C yaml library and headers installed, as well as the header files for your Python executables then you can use the non-roundtrip but faster C loader and emitter.

On Debian systems you should use:

sudo apt-get install libyaml-dev python-dev python3-dev

you can leave out python3-dev if you don’t use python3

For CentOS (7) based systems you should do:

sudo yum install libyaml-devel python-devel

Testing

Testing is done using tox, which uses virtualenv and pytest.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ruamel.yaml-0.10.14.tar.gz (234.8 kB view details)

Uploaded Source

Built Distributions

ruamel.yaml-0.10.14-cp34-none-win_amd64.whl (71.1 kB view details)

Uploaded CPython 3.4Windows x86-64

ruamel.yaml-0.10.14-cp34-none-win32.whl (71.1 kB view details)

Uploaded CPython 3.4Windows x86

ruamel.yaml-0.10.14-cp33-none-win_amd64.whl (71.1 kB view details)

Uploaded CPython 3.3Windows x86-64

ruamel.yaml-0.10.14-cp33-none-win32.whl (71.1 kB view details)

Uploaded CPython 3.3Windows x86

ruamel.yaml-0.10.14-cp27-none-win_amd64.whl (71.1 kB view details)

Uploaded CPython 2.7Windows x86-64

ruamel.yaml-0.10.14-cp27-none-win32.whl (137.7 kB view details)

Uploaded CPython 2.7Windows x86

ruamel.yaml-0.10.14-cp26-none-win_amd64.whl (71.1 kB view details)

Uploaded CPython 2.6Windows x86-64

ruamel.yaml-0.10.14-cp26-none-win32.whl (71.1 kB view details)

Uploaded CPython 2.6Windows x86

File details

Details for the file ruamel.yaml-0.10.14.tar.gz.

File metadata

  • Download URL: ruamel.yaml-0.10.14.tar.gz
  • Upload date:
  • Size: 234.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for ruamel.yaml-0.10.14.tar.gz
Algorithm Hash digest
SHA256 3f8b5522891c3d6e9cd5711651310d126b80dbb4f69a047fa4e9b20bf1f5c85f
MD5 10900e889a31fd184c8e6812f3e1eb94
BLAKE2b-256 4ce93b2a1299f88bf3176a35b445ce4e3b6cf2cd1c0dfc7cbea2446eb6a5f322

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp34-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp34-none-win_amd64.whl
Algorithm Hash digest
SHA256 ce9a081d1e1c42db4d3409a4cb778b5c356d3336b419b34afb03532cdda459ee
MD5 1fea48b569eb9237854480efab212a2f
BLAKE2b-256 605d21962500f2de3bba6113191cfe6554d944452237caf405d92fbdb994c58b

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp34-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp34-none-win32.whl
Algorithm Hash digest
SHA256 b5a77345f966722691eb59afc69a6354bd70d1dc1233fe5f8c7d29eca8b5af45
MD5 bb21e5ed0de27b66acf7457bedbe3e7f
BLAKE2b-256 c293d58482feafa12615a26264910c64bb40a388854363b33894acd76ef5ac67

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp33-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp33-none-win_amd64.whl
Algorithm Hash digest
SHA256 068095376ab06ed074f37eb423ccbef2110716509f42058fac1d1d32735f7dee
MD5 aaea933b8868c271583001dbe67cc98f
BLAKE2b-256 e3ce243ab55ef33b743edbb94e6a0c64e6dcba9c6233c748adf71a37359c7234

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp33-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp33-none-win32.whl
Algorithm Hash digest
SHA256 30f6010e41df348c5ff57457a3dc1ccd4c92974cf3142cac4309ea70b85da072
MD5 fcd4cda86d2f8e376478782c2df2cad3
BLAKE2b-256 e6d17f2c288b02ce65795a74a61ca4cc068c32964ceb9bbc28f1c6daa5cf2245

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp27-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp27-none-win_amd64.whl
Algorithm Hash digest
SHA256 76578642f48e9d7dfa3cd1d1cc023d6b827a0a823e1a60403f204a791df295d2
MD5 4d8fec4920d5b2113f675cf1fda85fd5
BLAKE2b-256 130cace30e73354912f60f6a6b061a525b252c09cbe95f4a6a7a795634368cee

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp27-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp27-none-win32.whl
Algorithm Hash digest
SHA256 a0b1b9075b600f931a8508f87c096b0fb33c78564d84a2bda8c6a225346ab4db
MD5 67a1d1dadfc04702fd4c62f4155905e3
BLAKE2b-256 01945d5b0d66684bce5ecf8dede6231607e769acdd65120411c84c7fd6808dc0

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp26-none-win_amd64.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp26-none-win_amd64.whl
Algorithm Hash digest
SHA256 4ced56f2415a35ffcbfa2d2f5ba1fffeeb45a8ca957429ee4c377005297f4523
MD5 79758bcf2417cca68165ecc7a3d002e0
BLAKE2b-256 3ca90b68affee5599a2cf41d9ccb193e419b321640c053459d8e8496e281e390

See more details on using hashes here.

File details

Details for the file ruamel.yaml-0.10.14-cp26-none-win32.whl.

File metadata

File hashes

Hashes for ruamel.yaml-0.10.14-cp26-none-win32.whl
Algorithm Hash digest
SHA256 64bcf117ec2f3b6e28979b12dfa3b0aa3694e40d1d7505380bdea4e19c8f38ff
MD5 797c0c93b59be0332531d1d21710b54b
BLAKE2b-256 0ed20022021b721b9ae0cd1da80d187fb4fe8e8fbadf68087cfb9c3497d1b289

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page