wildq 1.1.6

Command-line TOML/JSON/INI/YAML/XML processor using jq c bindings.

wildq - Command-line TOML/JSON/INI/YAML/XML/HCL processor using jq c bindings

Purpose of this package is to provide a simple wrapper arround jq for different formats. I'm tired of searching a package doing yaml jq, toml jq, ini jq etc. mainly used for scripting.

This script uses:

• @mwilliamson Python bindings on top of @stedolan famous jq lib
• swiss knife for coloration pygments
• binary built with pyinstaller
• easy CLI with click
• for supported types sources, check table Supported file types

Installation

Pip

pip install wildq

A binary is also available for different platform, pick one (both of wildq and wq will be in the archive)

MacOS

brew install ahmet2mir/tap/wildq
brew install ahmet2mir/tap/wq

GNU/Linux Binary

Compiled using glibc 2.17, it should work on lot of stable/LTS distros.

mkdir -p ~/bin/
curl -sL https://github.com/ahmet2mir/wildq/releases/download/v1.1.5/wildq-1.1.5-linux-x86_64.tar.gz -o wildq-1.1.5-linux-x86_64.tar.gz
tar xvfz wildq-1.1.5-linux-x86_64.tar.gz -C ~/bin
export PATH="~/bin:$PATH"

wq --help
wildq --help

Debian (no gpg signature) >= 10

curl -sL https://github.com/ahmet2mir/wildq/releases/download/v1.1.5/wildq_1.1.5-1_amd64.deb -o wildq_1.1.5-1_amd64.deb
sudo dpkg -i wildq_1.1.5-1_amd64.deb

wq --help
wildq --help

Centos (no gpg singature) >= 7

curl -sL https://github.com/ahmet2mir/wildq/releases/download/v1.1.5/wildq-1.1.5-1.x86_64.rpm -o wildq-1.1.5-1.x86_64.rpm
sudo yum install -y ./wildq-1.1.5-1.x86_64.rpm

wq --help
wildq --help

Windows

Wildq use jq.py and it's not yet available on windows platforms. I tried to compile it without windows machine and I failed, and I don't had the time to try to understand how Windows / C binding / Python works. If anybody would contribute, there is an open issue (jq and onigurama are 'compilable' on Windows so I think that someone confortable with that OS could make it);

Supported file types

type	color	ordering	output	source
hcl	json	no	json	pyhcl by @virtuald
ini	yes	no	yes	ConfigParser
json	yes	yes	yes	json
toml	yes	no	yes	toml by @uiri
xml	yes	no	yes	xmldict by @martinblech
yaml	yes	yes	yes	pyyaml

Usage

$ wildq -i help
Usage: wildq [OPTIONS] JQ_FILTER [FILE]

Options:
  -c, --compact-output            compact instead of pretty-printed output
  -r, --raw                       output raw strings, not content texts
  -C, --color-output              colorize content (default), mutally
                                  exclusive with --monochrome-output

  -M, --monochrome-output         monochrome (don't colorize content), mutally
                                  exclusive with --color-output

  --hcl                           Combine --input hcl --output json, mutally
                                  exclusive with other Combined options

  --ini                           Combine --input ini --output json, mutally
                                  exclusive with other Combined options

  --json                          Combine --input json --output json, mutally
                                  exclusive with other Combined options

  --toml                          Combine --input toml --output json, mutally
                                  exclusive with other Combined options

  --xml                           Combine --input xml --output json, mutally
                                  exclusive with other Combined options

  --yaml                          Combine --input yaml --output json, mutally
                                  exclusive with other Combined options

  -i, --input [hcl|ini|json|toml|xml|yaml]
                                  Define the content type of file, mutally
                                  exclusive with Combined option

  -o, --output [hcl|ini|json|toml|xml|yaml]
                                  Define the content type of printed output,
                                  mutally exclusive with Combined option
                                  (default input format)

  --help                          Show this message and exit.

For backward compatibility in previous version only --[yaml|json|toml|ini|xml|hcl] was possible with default to json output. We still keep Monochrome, raw and json output with thoses options. Output was similar to jq -MCr (no color, no compact and no quote on single value)

But now, by default it's colorized, not raw and if you specify input using -i or --input output will be the same format.

There is also a shorter command wq comming with the package.

Like jq cli, wildq supports both of stdin and file to the function

See examples to get some example.

Content of examples/json.json

{
    "general": {
        "user": "admin"
    },
    "keys": [
        {"key": "value1"},
        {"key": "value2"},
        "alone"
    ]
}

cat examples/json.json | wildq -i json ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

or

wildq -i json ".keys[]" examples/json.json
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

or

wq -i json ".keys[]" examples/json.json
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

For TOML

cat examples/toml.toml | wildq -i toml ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

For INI (no array)

cat examples/ini.ini | wildq -i ini ".keys"
{
    "key1": "value1",
    "key2": "value2"
}

For XML

cat examples/xml.xml | wildq -i xml "."
{
    "root": {
        "general": {
            "user": "admin"
        },
        "keys": {
            "element": [
                {
                    "key": "value1"
                },
                {
                    "key": "value2"
                },
                "alone"
            ]
        }
    }
}

For YAML

cat examples/yaml.yaml  | wildq -i yaml ".keys[]"
{
    "key1": "value1"
}
{
    "key2": "value2"
}
alone

For HCL

cat examples/hcl.hcl  | wildq -i hcl ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}

Tips and tricks

Loop on keys in bash without creating a subshell

wildq -i toml "keys[]" examples/toml.toml | while read -r key 
do
    echo "Getting key ${key}"
done

TODO

• add tests...
• add more control over filters and files
• use click for the CLI
• support different output
• detect automagically filetype
• support all jq types
• ordering

Contributing

Merge requests are welcome :)

License

Licensed under the terms of the Apache License, Version 2.0.

Repository URL

https://github.com/ahmet2mir/wildq