Skip to main content

jq for pythonista

Project description

jqfpy

Build Status

jq is too difficult

jq is too difficult, at least for me.

For example, extracting key-name when use is true only, from below JSON data.

{
  "apps": {
    "foo": {
      "use": true
    },
    "bar": {
      "use": true
    },
    "boo": {
      "use": true
    },
    "bee": {
      "use": false
    }
  }
}

What is jq's answer? (taking over 30 minutes, my past challenges).

$ cat data.json | jq '.apps | . as $$o | keys | map(select($$o[.].use))'
[
  "bar",
  "boo",
  "foo"
]

If you have python's knowledge, this is tiny oneliner, isn't it?

$ cat data.json | jqfpy '[k for k, opts in get("apps").items() if opts["use"]]'
[
  "foo",
  "bar",
  "boo"
]

(get() is special function, like a json.load(sys.stdin).get.)

install

$ pip install jqfpy

how to use

describe syntax

todo.

tutorial

this is jqfpy version of jq's Tutorial

$ alias jsonDATA="curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5'"
# jq.
$ jsonDATA | jq '.'
# jqfpy.
$ jsonDATA | jqfpy 'get()'
# jq.
$ jsonDATA | jq '.[0]'
# jqfpy.
$ jsonDATA | jqfpy 'get()[0]'
# jq.
$ jsonDATA | jq '.[0] | {message: .commit.message, name: .commit.committer.name}'
# jqfpy.
$ jsonDATA | jqfpy 'd = get()[0]; {"message": get("commit/message", d), "name": get("commit/committer/name", d)}'
# or
$ jsonDATA | jqfpy '{"message": get("0/commit/message"), "name": get("0/commit/committer/name")}'
# jq.
$ jsonDATA | jq '.[] | {message: .commit.message, name: .commit.committer.name}'
# jqfpy.
$ jsonDATA | jqfpy --squash 'L = get(); [{"message": get("commit/message", d), "name": get("commit/committer/name", d)} for d in L]'
# jq.
$ jsonDATA | jq '[.[] | {message: .commit.message, name: .commit.committer.name, parents: [.parents[].html_url]}]'
# jqfpy.
$ jsonDATA | 'L = get(); [{"message": get("commit/message", d), "name": get("commit/committer/name", d), "parents": [p["html_url"] for p in d["parents"]]} for d in L]'
# or (using h.pick)
$ jsonDATA | 'L = get(); [h.pick("commit/message@message", "commit/committer/name@name", "parents[]/html_url@parents", d=d) for d in L]'

additionals

other formats support

jqfpy is supporting other formats(but this is experimental feature)

  • yaml
  • ltsv

if you want to use yaml supported version. install via below command.

$ pip install jqfpy[yaml]

and calling jqfpy with --input-format,-i option and --output-format,-o option.

02data.yaml

person:
  name: foo
  age: 20
  nickname: fool
$ cat 02data.yaml | jqfpy -i yaml 'get("person")'
{
  "name": "foo",
  "age": 20,
  "nickname": "fool"
}

$ cat 02data.yaml | jqfpy -i yaml -o ltsv 'get("person")'
name:foo	age:20	nickname:fool

helper functions

helper functions are included.

  • pick()
  • omit()
  • flatten()
  • chunk()
  • loadfile()
  • dumpfile()

pick()

$ cat 02data.yaml | jqfpy -i yaml 'h.pick("person/name", "person/age")'
{
  "person": {
    "name": "foo",
    "age": 20
  }
}

$ cat 02data.yaml | jqfpy -i yaml 'h.pick("person/name@name", "person/age@age")'
{
  "name": "foo",
  "age": 20
}

omit()

$ cat 02data.yaml | jqfpy -i yaml 'h.omit("person/nickname")'
{
  "person": {
    "name": "foo",
    "age": 20
  }
}

flatten()

$ seq 1 5 | jqfpy --slurp -c 'L = get(); [L, L]'
[[1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]
$ seq 1 5 | jqfpy --slurp -c 'L = get(); h.flatten([L, L], n=1)'
[1, 2, 3, 4, 5, 1, 2, 3, 4, 5]

chunk()

$ seq 1 10 | jqfpy --slurp -c 'h.chunk(get(), n=3)'
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10]]

loadfile()

$ ls *.json
a.json b.json
$ echo '["a", "b"]' | jqfpy '{name: h.loadfile(f"{name}.json") for name in get()}'

--here and --relative-path options.

# see ./x.json
$ jqfpy 'h.loadfile("x.json")' a/b/main.json

# see ./a/b/x.json
$ jqfpy --here a/b/ 'h.loadfile("x.json")' a/b/main.json

# see ./a/b/x.json
$ jqfpy --relative-path 'h.loadfile("x.json")' a/b/main.json

dumpfile()

$ echo {"person0.json": {"name": "foo", "age": 20}, "person1.json": {"name": "bar}} | jqfpy '[h.dumpfile(v, fname) for fname, v in get().item()]' > /dev/null

individual helper module with --additionals

match.py

import re


def match(rx, text):
    if text is None:
        return False
    return re.search(rx, text)
$ cat examples/additionals/00data.json | jqfpy --additionals=./match.py '[d for d in get("constraint") if h.match("^1\..+", d.get("version"))]'
[
  {
    "name": "github.com/Masterminds/vcs",
    "version": "1.11.0"
  },
  {
    "name": "github.com/boltdb/bolt",
    "version": "1.0.0"
  }
]

0.6.1

  • fix --show-none option, always ignored, in 0.6.0

0.6.0

  • add --show-none (default, if return value is None, doesn't write to stdout)
  • add raw format
  • add short option -a and -u (drop --ascii-output asa -a)

0.5.4

  • change arguments order, h.dumpfile (#41)
  • fix bug for get() (#37)

0.5.3

  • fix bug for h.dumpfile()

0.5.2

  • add h.loadfile() function (#35)

0.5.1

  • h.dumpfile() with raw option

0.5.0

  • drop python 3.5
  • yaml output, oneline string including hash, always quoted (#28)
  • treat '/foo' as 'foo' (lstripped)
  • add h.dumpfile() function (#31)

0.4.2

  • support python 3.4
  • custom additionals (--additionals option)
  • new helpers -- h.flatten, h.flatten1 and h.chunk

0.4.1

  • ordered is default, when json loading

0.4.0

  • experimental ltsv support
  • adding helper functions (h.omit(), h.pick())
  • extend get() function's function
  • rename option(--show-code-only to --show-code)

0.3.2

  • the future is dropped that showing help when calling with no arguments

0.3.0

  • experimental yaml format support

0.2.0

  • correct behaviours of --slurp and --unbuffered
  • support accessing data by json pointer like format
  • compact output support
  • multiple files support
  • exec code only once

0.1.0

  • adding some options
  • nocode is same as jq . (js == jq .)
  • showing pycode when error is raised

0.0.1

  • first release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for jqfpy, version 0.6.1
Filename, size & hash File type Python version Upload date
jqfpy-0.6.1-py2.py3-none-any.whl (12.9 kB) View hashes Wheel py2.py3
jqfpy-0.6.1.tar.gz (12.5 kB) View hashes Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page