yamlfixer-opt-nc 0.8.0

automates the fixing of problems reported by yamllint

yamlfixer

❔ About

yamlfixer automates the fixing of problems reported by yamllint by parsing its output.

🎬 Demo

Click on the white triangle in the image below to view a short video demonstration:

📑 Installation

💡 You can try the install process online thanks to the dedicated Katacoda scenario.

The easiest way to install yamlfixer is from pypi, as described below.

🐧 Linux install

python3 -m pip install yamlfixer-opt-nc

🪟 Windows install

python -m pip install yamlfixer-opt-nc

pipx setup

For an optimal experience we recommand using pipx.

To install :

pipx install yamlfixer-opt-nc
pipx list

To upgrade :

pipx upgrade yamlfixer-opt-nc

To uninstall :

pipx uninstall yamlfixer-opt-nc

🚀 Usage

This software automatically fixes some errors and warnings reported by yamllint.

usage: yamlfixer [-h] [-v] [-b] [-B BACKUPSUFFIX] [-d] [-D DIFF_FILE] [-e EXTENSIONS] [-f]
                 [-l] [-N] [-n] [-r LEVEL] [-j | -p | -s] [-t TABSIZE]
                 [-c CONFIG_FILE | -C CONFIG_DATA]
                 [FILE_or_DIR [FILE_or_DIR ...]]

Fix formatting problems in YAML documents. If no file is specified, then reads input from `stdin`.

positional arguments:
  FILE_or_DIR           the YAML files to fix. Use `-` to read from `stdin`.

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         display this program's version number and exit.
  -b, --backup          make a backup copy of original files.
  -B BACKUPSUFFIX, --backupsuffix BACKUPSUFFIX
                        sets the suffix for backup files, `.orig` is the default.
  -d, --debug           output debug information to stderr.
  -D DIFF_FILE, --diffto DIFF_FILE
                        name of the file a unified diff will be written to.
                        Defaults to `/dev/null`.
  -e EXTENSIONS, --ext EXTENSIONS
                        comma separated list of acceptable extensions when searching directories
                        for YAML files. Defaults to `yaml,yml,yamllint`.
  -f, --forcecolors     force colored output even if stream is not a TTY.
  -l, --listfixers      output the list of available fixers.
  -N, --nosyntax        don't try to fix syntax errors.
  -n, --nochange        don't modify anything.
  -r LEVEL, --recurse LEVEL
                        sets the maximum recursion level for directories. Default is `0` meaning
                        no recursion, and any negative value means no limit.
  -j, --jsonsummary     output JSON summary to stderr.
  -p, --plainsummary    output plain text summary to stderr.
  -s, --summary         output colored plain text summary to stderr. If stderr is not a TTY
                        output is identical to --plainsummary.
  -t TABSIZE, --tabsize TABSIZE
                        sets the number of spaces to replace tabs with, default is `2`.
  -c CONFIG_FILE, --config-file CONFIG_FILE
                        path to yamllint's custom configuration file, none by default.
  -C CONFIG_DATA, --config-data CONFIG_DATA
                        custom configuration for yamllint as YAML source, none by default.

yamlfixer launches yamllint on each specified filename, then parses its output and tries to fix the reported problems. The special filename - means stdin, and is assumed if there's no other filename argument.

If input is read from stdin, the corrected output will be sent to stdout.

Other files will be overwritten if needed. Original files, stdin excepted, can be preserved as .orig if the --backup command line option is used. You can specify any other backup filename suffix with the --backupsuffix command line option.

Both summaries and diagnostic information are sent to stderr.

This command exits with status 2 if there are incompatible command line options. It exits with -2 if yamllint is not available on your system. Otherwise it exits with 0 if all input files either are skipped, entirely fixed, or already successfully passed yamllint strict mode before, else -1.

For convenience, all or parts of the command line arguments can be read from a file, one per line, by using the well known @argsfile syntax. For example you could do something like this :

$ find . -type f -name "*.yml" >list-of-yaml-files
$ yamlfixer --nochange --summary @list-of-yaml-files

Although this could probably be shortened to :

$ yamlfixer --nochange --summary --recurse -1 .

IMPORTANT: Not all problems are fixable by yamlfixer. Due to the fact that yamllint doesn't currently report all faulty lines, yamlfixer might even introduce indentation problems under some circumstances.

⚠️Use at your own risk, you have been warned... ⚠️

Fixers

yamlfixer currently (as of 0.6.4) can fix the following problems as reported by yamllint :

• comment not indented like content (comments-indentation)
• line too long
• missing document start
• missing starting space in comment (comments)
• no new line character at the end of file
• syntax error: expected '<document start>', but found '<stream end>' (syntax)
• syntax error: expected <block end>, but found '<block mapping start>'
• syntax error: expected <block end>, but found '<block sequence start>' (syntax)
• syntax error: expected <block end>, but found '?'
• syntax error: found character '\t' that cannot start any token (syntax)
• syntax error: mapping values are not allowed here
• too few spaces after comma (commas)
• too few spaces before comment (comments)
• too many blank lines
• too many spaces after colon (colons)
• too many spaces after comma (commas)
• too many spaces after hyphen (hyphens)
• too many spaces before colon (colons)
• too many spaces before comma (commas)
• too many spaces inside braces (braces)
• too many spaces inside brackets (brackets)
• too many spaces inside empty brackets (brackets)
• trailing spaces (trailing-spaces)
• truthy value should be one of [false, true] (truthy)
• wrong indentation: expected

An always up-to-date list of fixers can be obtained with yamlfixer --listfixers.

IMPORTANT : fixing syntax errors is done on a best effort basis and may work only partially or not at all for you. Use the -N|--nosyntax command line switch do prevent yamlfixer from trying to fix syntax errors.

💪 Tips and tricks

Find here a set of tips & tricks about how to achieve great things.

Don't find the usecase you're looking for ➡️ 🎫 Fill a dedicated issue so we could share your idea with the comunity

⏩ One liners

Most of use love short and efficient command lines. Here are some ready to use ones :

Piping json summary through jq

yamlfixer --jsonsummary bad.yml 2>&1 | jq

So you can get a nicely colored (and validated json output) :

  "filestofix": 1,
  "passedstrictmode": 1,
  "modified": 0,
  "fixed": 0,
  "skipped": 0,
  "notwriteable": 0,
  "unknown": 0,
  "details": {
    "/home/jerome/yamlfixer/bad.yml": {
      "status": "PASSED",
      "issues": 0,
      "handled": 0
    }
  },
  "nochangemode": false
}

Check if yamlfixer could fix a yaml and get the exit code

See how piping fixing and linting can be interesting... and get exit code so you can go further in automation :

yamlfixer <bad.yml | yamllint --format parsable --strict -
echo $?

🧰 Single purpose tools worth knowing

• ytt : "YAML templating tool that works on YAML structure (instead of text)."
• jq : "lightweight and flexible command-line JSON processor."
• vimdiff : "edit two, three or four versions of a file with Vim and show differences"
• icdiff : "improved colored diff "
• gomplate : "A flexible commandline tool for template rendering. Supports lots of local and remote datasources."

🔖 Related contents

• Dedicated Post explaining how we are using this project to automate yaml linting and fixing
• GH Action relying on this project
• Dedicated Katacoda scenario so you can see it live
• Yamlfixer video intro (French, 15')
• From cli to community DEV.to blog post to better understand yamlfixer inception, historyand roadmap

📖 Licensing information

Copyright (C) 2021-2022 OPT-NC

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

🧑‍🤝‍🧑 Contact

To contact the authors of this software, simply fill an issue on this project.

OPT-NC, aka Office des Postes et Télécommunications de Nouvelle-Calédonie, check OPT-NC Github Organization page for more.