Skip to main content

Command-line TOML/JSON/INI/YAML/XML processor using jq c bindings.

Project description

wildq - Command-line TOML/JSON/INI/YAML/XML/HCL processor using jq c bindings

GitHub image Build Status

Purpose of this package is to provide a simple wrapper arround jq for different formats. I'm tired of searching a package doing yaml jq, toml jq, ini jq etc. mainly used for scripting.

This script uses:

  • @mwilliamson Python bindings on top of @stedolan famous jq lib
  • swiss knife for coloration pygments
  • binary built with pyinstaller
  • easy CLI with click
  • for supported types sources, check table Supported file types

Installation

Pip

pip install wildq

Binary

A binary is also available for different platform, pick one (both of wildq and wq will be in the archive)

GNU/Linux

curl https://github.com/ahmet2mir/wildq/archive/1.0.6.zip -o wildq.zip
unzip wildq.zip -d ~~/bin~~
rm -f wildq.zip

MacOS

curl https://github.com/ahmet2mir/wildq/archive/1.0.6.zip -o wildq.zip
unzip wildq.zip -d ~/bin~
rm -f wildq.zip

Windows

Wildq use jq.py and it's not yet available on windows platforms. I tried to compile it without windows machine and I failed, and I don't had the time to try to understand how Windows / C binding / Python works. If anybody would contribute, there is an open issue (jq and onigurama are 'compilable' on Windows so I think that someone confortable with that OS could make it);

Supported file types

type color ordering output source
hcl json no json pyhcl by @virtuald
ini yes no yes ConfigParser
json yes yes yes json
toml yes no yes toml by @uiri
xml yes no yes xmldict by @martinblech
yaml yes yes yes pyyaml

Usage

$ wildq -i help
Usage: wildq [OPTIONS] [FILE] JQ_FILTER

Options:
  -c, --compact-output            compact instead of pretty-printed output
  -r, --raw                       output raw strings, not content texts
  -C, --color-output              colorize content (default), mutally
                                  exclusive with --monochrome-output

  -M, --monochrome-output         monochrome (don't colorize content), mutally
                                  exclusive with --color-output

  --hcl                           Combine --input hcl --output json, mutally
                                  exclusive with other Combined options

  --ini                           Combine --input ini --output json, mutally
                                  exclusive with other Combined options

  --json                          Combine --input json --output json, mutally
                                  exclusive with other Combined options

  --toml                          Combine --input toml --output json, mutally
                                  exclusive with other Combined options

  --xml                           Combine --input xml --output json, mutally
                                  exclusive with other Combined options

  --yaml                          Combine --input yaml --output json, mutally
                                  exclusive with other Combined options

  -i, --input [hcl|ini|json|toml|xml|yaml]
                                  Define the content type of file, mutally
                                  exclusive with Combined option

  -o, --output [hcl|ini|json|toml|xml|yaml]
                                  Define the content type of printed output,
                                  mutally exclusive with Combined option
                                  (default input format)

  --help                          Show this message and exit.

For backward compatibility in previous version only --[yaml|json|toml|ini|xml|hcl] was possible with default to json output. We still keep Monochrome, raw and json output with thoses options. Output was similar to jq -MCr (no color, no compact and no quote on single value)

But now, by default it's colorized, not raw and if you specify input using -i or --input output will be the same format.

There is also a shorter command wq comming with the package.

Like jq cli, wildq supports both of stdin and file to the function

See examples to get some example.

Content of examples/json.json

{
    "general": {
        "user": "admin"
    },
    "keys": [
        {"key": "value1"},
        {"key": "value2"},
        "alone"
    ]
}
cat examples/json.json | wildq -i json ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

or

wildq -i json ".keys[]" examples/json.json
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

or

wq -i json ".keys[]" examples/json.json
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

For TOML

cat examples/toml.toml | wildq -i toml ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

For INI (no array)

cat examples/ini.ini | wildq -i ini ".keys"
{
    "key1": "value1",
    "key2": "value2"
}

For XML

cat examples/xml.xml | wildq -i xml "."
{
    "root": {
        "general": {
            "user": "admin"
        },
        "keys": {
            "element": [
                {
                    "key": "value1"
                },
                {
                    "key": "value2"
                },
                "alone"
            ]
        }
    }
}

For YAML

cat examples/yaml.yaml  | wildq -i yaml ".keys[]"
{
    "key1": "value1"
}
{
    "key2": "value2"
}
alone

For HCL

cat examples/hcl.hcl  | wildq -i hcl ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}

Tips and tricks

Loop on keys in bash without creating a subshell

wildq -i toml "keys[]" examples/toml.toml | while read -r key 
do
    echo "Getting key ${key}"
done

TODO

  • add tests...
  • add more control over filters and files
  • use click for the CLI
  • support different output
  • detect automagically filetype
  • support all jq types
  • ordering

Contributing

Merge requests are welcome :)

License

Licensed under the terms of the Apache License, Version 2.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

wildq-1.1.0-py3-none-any.whl (9.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page