ytools
Command-line tool and python module for …
validating json and yaml files against a json-schema in json or yaml format
selectively dumping nodes from yaml (or json) documents in yaml or json format.
Features
Output yaml as json or python
Output json as yaml or python (provided that there are no duplicate mapping entry in the json source)
Extract particular nodes from yaml and json files.
If yaml is used as output format (default) the output is a valid yaml document.
Validate yaml and json documents.
The json-schema can be provided in yaml format as well, which improves readability and writability.
Preserve order of mapping-keys in yaml and json output.
Multi-document support
Multiple input files
… as well as multiple yaml documents within a file
… and a combination of both
Installation
pip install ytools
Description
python ytools.py input.json … for converting json to yaml, or …
python ytools.py input.yaml -f json … for the opposite direction.
ytools -h
$ ytools -h
Usage: /usr/local/bin/ytools [OPTION] -p JSONPATH_EXPRESSION FILE...
Dumps data from json (or yaml) documents in yaml format. Command line wrapper
for jsonpath-ng.
Options:
-h, --help show this help message and exit
-p PATH, --json-path=PATH
Syntax for jsonpath expression:
https://pypi.python.org/pypi/jsonpath-ng/1.4.2
-f OUTPUTFORMAT, --output-format=OUTPUTFORMAT
Output format. Can be "yaml", "json" or "python".
[default: yaml]
-y YAML_OPTIONS, --yaml-options=YAML_OPTIONS
kwargs for yaml.dump (pyYaml) as yaml.mapping (for
experts). [default: '{explicit_start: True,
explicit_end: True, allow_unicode: True}']
-j JSON_OPTIONS, --json-options=JSON_OPTIONS
kwargs for json.dumps as yaml.mapping (for experts).
[default: '{indent: 2, encoding: utf-8}']
-v SCHEMA, --validate=SCHEMA
Validate documents against json-schema
--encoding=ENCODING Set encoding of input documents (if different from
utf-8)
Command Line Samples
The samples are based on the following data.
Sample Data
Input Data
input.yaml:
documents:
- title: Some document title
sections:
- title: Some section title
description: Some section description
text: Some text for some section
chapters:
- title: Some chapter title
description: Some chapter description
text: The text of some chapter
- title: Some other chapter title
description:
- descriptionparagraph1: Some description for other chapter
- descriptionparagraph2: Some description for other chapter
text: The text of some other chapter
- title: Some other section title
description: Some other section description
text: Some text for some other section
chapters:
- title: About encoding
description: "Some German: äöü,ÄÖÜ,ß"
Schema for validating input.yaml
schema.yaml
$schema: "http://json-schema.org/schema#"
definitions:
chapter:
type: object
properties:
title: {type: string}
description:
oneOf: [{type: string}, {type: array}]
text: {type: string}
additionalProperties: false
required: [title, description]
chapters:
type: array
items: {$ref: "#/definitions/chapter"}
additionalItems: false
section:
type: object
properties:
title: {type: string}
description: {type: string}
text: {type: string}
chapters: {$ref: "#/definitions/chapters"}
additionalProperties: false
required: [title, description]
sections:
type: array
items: {$ref: "#/definitions/section"}
additionalItems: false
document:
type: object
properties:
title: {type: string}
description: {type: string}
sections: {$ref: "#/definitions/sections"}
additionalProperties: false
required: [title, description]
documents:
type: array
items: {$ref: "#/definitions/document"}
additionalItems: false
type: object
properties:
documents: {$ref: "#/definitions/documents"}
additionalProperties: false
Outputs
When not providing a jsonpath expression
If you don’t provide a jsonpath expression using the -p option ytools uses '$' as default and therefore dumps the complete input:
🎼 ytools input.yaml
---
documents:
- title: Some document title
description: The document's description
sections:
- title: Some section title
description: Some section description
text: Some text for some section
chapters:
- {title: Some chapter title, description: Some chapter description, text: The
text of some chapter}
- title: Some other chapter title
description:
- {descriptionparagraph1: Some description for other chapter}
- {descriptionparagraph2: Some description for other chapter}
text: The text of some other chapter
- title: Some other section title
description: Some other section description
text: Some text for some other section
chapters:
- {title: About encoding, description: 'Some German: äöü,ÄÖÜ,ß'}
...
Output format yaml (default)
With the yaml output format by default, each match is output as a separate yaml document. This way we achieve that the output is valid yaml.:
🎼 ytools -p '$..chapters[*].description' input.yaml
--- Some chapter description
...
---
- {descriptionparagraph1: Some description for other chapter}
- {descriptionparagraph2: Some description for other chapter}
...
--- 'Some German: äöü,ÄÖÜ,ß'
...
If you want different behavior you can set explicit_start and/or explicit_end to False. In this case the output will no longer be guaranteed to be valid yaml:
🎼 ytools -p '$..chapters[*].description' input.yaml --yaml-options='{explicit_start: False, explicit_end: False}'
Some chapter description
...
- {descriptionparagraph1: Some description for other chapter}
- {descriptionparagraph2: Some description for other chapter}
'Some German: äöü,ÄÖÜ,ß'
Other output formats
Unfortunately, when using the json or python output format the same selection can’t produce valid json or python output. That’s because neither json nor python support the concept of (multiple) documents:
🎼 ytools -p '$..chapters[*].description' input.yaml -f json --json-options='{indent: 4}'
"Some chapter description"
[
{
"descriptionparagraph1": "Some description for other chapter"
},
{
"descriptionparagraph2": "Some description for other chapter"
}
]
"Some German: \u00e4\u00f6\u00fc,\u00c4\u00d6Y,\u00df"
That’s definitely not valid json.
Neither is the following valid python:
🎼 ytools -p '$..chapters[*].description' input.yaml -f python
Some chapter description
[{'descriptionparagraph1': 'Some description for other chapter'}, {'descriptionparagraph2': 'Some description for other chapter'}]
Some German: äöü,ÄÖÜ,ß
So if you ever want to process the output automatically please stick to ``yaml``.
Python Samples
Sample Data
Input Data
test/sampledata.yaml
name: 'my_name'
date: '2017-10-01T10:55:00Z'
metrics:
percentage:
value: 87
trend: stable
Schema for Validating
test/sampleschema.yaml
type: object
properties:
name: { type: string }
date: { type: string, format: date-time }
metrics:
type: object
properties:
percentage:
type: object
properties:
value:
type: number
minimum: 0
maximum: 100
trend: { type: string, enum: [down, stable, up] }
additionalProperties: false
required: [value, trend]
additionalProperties: false
required: [percentage]
additionalProperties: false
required: [name, date, metrics]
Validation
ytools.validate("test/sampleschema.yaml", ["test/sampledata.yaml"])
… will not output anything because of successful validation. Play around if you want to see failing validation - it’s quite easy to make it fail ;-)
Dumping
ytools.dump("test/sampledata.yaml", "$.metrics", yaml_options="default_flow_style: false")
… will output …
--- percentage: value: 87 trend: stable ...
Metadata
Release files for ytools 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ytools-1.0.0.tar.gz | 11.6 kB | Without provenance | Details for ytools-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ytools-1.0.0-py2-none-any.whl | Python 2 | none | any | Details for ytools-1.0.0-py2-none-any.whl |
Total release size: 21.8 kB
Release files / ytools-1.0.0.tar.gz
| Download URL | ytools-1.0.0.tar.gz |
|---|---|
| Size | 11.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
722022c6b7aa7a7f7878918d802c45d7a3c569340c167eb54faae92cc1339c7f
|
|
BLAKE2b-256 checksum How to use checksums |
95e2569d41b33d614772556d61876a9bb7c4b7f5f49c64c420fc1574ddca2349
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.11.0 pkginfo/1.4.2 requests/2.18.4 setuptools/33.1.1 requests-toolbelt/0.8.0 tqdm/4.23.4 CPython/2.7.14
|
Release files / ytools-1.0.0-py2-none-any.whl
| Download URL | ytools-1.0.0-py2-none-any.whl |
|---|---|
| Size | 10.2 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
9622f6436f3954d095019c6b7e9f2de05bf721b164a7233024ad22d6e3ef23ca
|
|
BLAKE2b-256 checksum How to use checksums |
c557d4275620dc28546cb30a2622eaca6c56441aeeaf0790e9b84b4c69f974fe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.11.0 pkginfo/1.4.2 requests/2.18.4 setuptools/33.1.1 requests-toolbelt/0.8.0 tqdm/4.23.4 CPython/2.7.14
|
Security for ytools 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.