Replacement is a python utility that parses a yaml template and outputs text.
Project description
Replacement
Replacement is a python utility that parses a yaml template and outputs text.
Installing
pip3 install replacement
or, if you use nix:
nix-env --install -A 'nixpkgs.python36Packages.replacement'
Introduction
NOTE: the examples given here are further documented in the tests.
A template
is a YAML file containing a replacement
object.
A replacement object contains a list of blocks
.
1. basic template
A template:
---
# simplistic template example
# tests/hello.yaml
replacement:
- text: text
input:
hello world
...
Execute a template using replacement
:
$ replacement -t tests/hello.yaml
hello world
Blocks always begin with a directive
, which specifies:
- the data type the block will
yield
, such astext
or adict
- the input data type, such as
file
oryaml
In the block above, text: text
is the directive;
it specifies the block takes text
input and yields text
output.
2. reading from a file
File hello.out
:
hello world
---
# read from file
# tests/file.yaml
replacement:
- text: file
input:
hello.out
...
$ replacement -t tests/file.yaml
hello world
Notice that:
- The directive in the above block is
text: file
; this means: "yield text, input from a file". - The
input
key is used for the filename to be read; in the first exampleinput
was used to store the literal text input. - File paths are relative to the path of the template.
intermission: schema
A schema of all the replacement directives is contained in schema.yaml.
Here is a snippet showing just the directive
and input
portions of a block:
---
##
# format of a block
##
block:
- yield: inputType # this is a directive
input: inputVal
##
# definition of schema elements
##
schema:
# 'yield' : what to output (how to format output)
yield:
- text # text: a list of newline-separated strings
- dict # a dictionary of key-value pairs
- meta # output nothing: set a metadata (substitution dictionary) variable
# 'input' : where to source data
inputType:
# we can input everything we output (see 'yield' above)
- text
- dict
- meta # a key-value pair retrieved from substitutions dictionary
# we can also input from other sources
- file # open a file on disk
- eval # python3 eval() statement
- func # import and then call call a function
- exec # subprocess execution (usually bash)
inputVal: input_specific
...
3. metadata and substitution
The meta
directive specifies that the output of a block should be inserted
into the "substitutions dictionary" (aka: meta
).
- this implies input must be a valid dictionary
- replacement will parse YAML or JSON (which is a subset of YAML)
The proc
keyword specifies that block output should be processed
(string substitution).
Valid proc
directives are:
format
::str.format()
substitute
::string.Template().substitute()
safe_substitute
::string.Template().safe_substitute()
---
# metadata
# tests/metadata.yaml
replacement:
# parse 'input' and insert the resulting dictionary into 'meta'
- meta: dict
# metadata may be given as an object
input:
version: 1.1
tag: my_awesome_tag
- meta: text
# metadata may be given as text which can be parsed as valid YAML
input: |
---
message: hello world
...
- meta: text
# metadata may also be given as JSON
input: |
{ "hi": 5 }
# use 'proc' to specify that 'str.format(**meta)' should be run on output
- text: text
proc: format
input: |
v{version} tag "{tag}"
- text: text
proc: substitute
input: |
message $message
- text: text
proc: safe_substitute
input: |
hi $hi
this value may not exist - $nonexistent
...
$ replacement -t tests/metadata.yaml
v1.1 tag "my_awesome_tag"
message hello world
hi 5
this value may not exist - $nonexistent
Metadata can also be read from a file.
File a.json
:
{
"hello": "world",
"hi": 5,
"list": [ 1, 2, 3, 4 ]
}
---
# metadata
# tests/meta_file.yaml
replacement:
# parse file, inserting result into metadata dictionary
- meta: file
input:
a.json
# string substitution with 'proc'
- text: text
proc: format
input: |
hello {hello}
hi {hi}
list {list}
...
$ replacement -t tests/meta_file.yaml
hello world
hi 5
list [1, 2, 3, 4]
4. nesting
- Blocks are executed in sequence, and may nest.
- nested blocks may extend/alter
meta
, which will be seen by later blocks or child blocks, but not parent blocks.
- nested blocks may extend/alter
---
# nesting and dictionaries
# tests/nesting.yaml
replacement:
# parse 'input' and insert the resulting dictionary into 'meta'
- meta: dict
input:
version: 1.1
# same thing, but parse *text* input, instead of a dictionary
- meta: text
input: | # note the '|'
---
tag: my_awesome_tag
...
# use 'proc' to specify that 'str.format(**meta)' should be run on output
- text: text
proc: format
input: |
v{version} tag "{tag}"
- text: text
input:
# metadata additions/changes seen by later blocks and any children,
# but seen outside this list
- meta: dict
input:
version: 1.0
- text: text
proc: format
input: |
#v{version} tag "{tag}" (version clobbered in inner scope)
- text: file
input:
hello.out # contains 'hello world'
- text: text
proc: format
input: |
outer still v{version}
...
$ replacement -t tests/nesting.yaml
v1.1 tag "my_awesome_tag"
#v1.0 tag "my_awesome_tag" (version clobbered in inner scope)
hello world
outer still v1.1
5. preprocessing
Substitution can also be performed on the values of the block itself
before it is parsed.
The keyword prep
is used, with the same semantics as proc
(above):
---
# preprocessing
# tests/prep.yaml
replacement:
- meta: dict
input:
filename: hello.out
# preprocessing will substitute {filename} before evaluating 'file' input
- text: file
prep: format
input: |
{filename}
...
$ replacement -t tests/prep.yaml
hello world
6. access to python eval
---
# use of eval
# tests/an_eval.yaml
replacement:
# 'eval' returning a dictionary that can me appended to 'meta'
- meta: eval
input: |
{"hello": 5 + 1}
- text: text
prep: format
input: |
hello {hello}
# eval returning a scalar value
- text: eval
prep: format
input: |
{hello}**3
...
$ replacement -t tests/an_eval.yaml
hello 6
216
7. imports and function execution
The func
input directive can be used to find and call an external function
(see demo.py for reference):
---
# function execution
# tests/func.yaml
replacement:
# run a function returning a dictionary, and merge that into 'meta'
- meta: func
args:
existing: {'original': 'thesis'}
# NOTE that this is sensitive to PYTHONPATH
input: |
tests.demo.ret_a_dict
- text: text
prep: format
input: |
original {original}
secret {secret}
# run a function returning a dictionary and export return as JSON
- text: func
options:
- json # emit JSON rather than YAML (the default)
args:
existing: {'original': 'thesis'}
input: |
tests.demo.ret_a_dict
# run a function returning a list of strings; treat each one as a line of text
- text: func
args: {}
input: |
tests.demo.ret_a_stream
...
$ replacement -t tests/an_eval.yaml
original thesis
secret 42
{"secret": 42, "original": "thesis"}
1. hello
2. world
Project TODO
Project TODO list
-
subprocess execution and output capture
-
Packaging:
- proper test runner (perhaps tox?)
- add test runner to all the builds
-
Dependency output command: runs all preprocessing and outputs a list of file dependencies. For use e.g. in Makefiles.
-
Express the schema formally and write validation code for it.
-
This README itself becomes a template, automatically including:
- test YAML files
- BASH output of tests
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for replacement-0.2.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 846bbbe230c0a4ef0144c904d6c172fe5b3146497a5ac23b7a792652b8e07900 |
|
MD5 | 1595ddc259b75cf2a2eefc2759193b03 |
|
BLAKE2b-256 | 444f3902b62ddc887f4aa00f6a312926cf59e2ac9654f2ced67e05c0433fa380 |