jqson 0.2.0

Define and run test queries from JSON.

JqSON

The JqSON library was developed to provide a simple way to define queries in JSON format, which can later be run in Python. It can be utilized in testing automations, where expected output of a system can be submitted a predefined set of tests.

from dataclasses import dataclass
from jqson import Query

@dataclass
class Person(object):
    name: str = ""
    age: int = 0
    children: list = ()

P1 = Person(name="P1", age=10)
P2 = Person(name="P2", age=7)
P3 = Person(name="P3", age=50, children=[P1, P2])
P4 = Person(name="P4", age=77, children=[P3])

text = """
[
    {"query": "where", "path": [
        {"query": "not_empty", "left": "children"}
    ]},
    {"query": "all", "path": [
        {"query": "attr", "name":"age"},
        {"query": "and", "queries": [
            {"query": "not_null"},
            {"query": ">", "value": 20},
            {"query": "<", "value": 100}
        ]}
    ]}
]
"""

data = [P1, P2, P3, P4]
query = Query.from_text(text)
print(query(data))

# equivalent of
print(all((p.age is not None and 20 < p.age < 100) for p in data if p.children))

Installation

The JqSON library is fully implemented in Python. No additional compiler is necessary. After downloading the source code just run the following command from the jqson folder:

$ python setup.py install

or simply by using pip

$ pip install jqson

Requirements

• Python 3.11+

Documentation

Selector Queries

To retrieve a value from the input data, the following selector queries can be used:

• Attr: {"query": "attr", "name": "NAME"} is equivalent to getattr(input_data, NAME)
• Item: {"query": "item", "key": "KEY"} is equivalent to input_data[KEY]
• Bool: {"query": "bool"} is equivalent to bool(input_data)
• Len: {"query": "len"} is equivalent to len(input_data)

Path Query

One of the main query type is the path query {"query": "path", "path": "PATH"}, which allows to define a sequence of queries to be executed in order. The input data are processed by the first query and its output is passed to the next, and so on. This selector is frequently used as an attribute of other queries, which require a value to be retrieved. The PATH can be specified as a single selector query or as a list of individual queries.

{"query": "path", "path":
  [
    {"query": "attr", "name": "address"},
    {"query": "attr", "name": "city"},
    {"query": "==", "value": "Ankh-Morpork"}
  ]
}

For convenience, a path can also be automatically converted directly from a string, following this syntax:

• {"query": "path", "path": "age"} is equivalent to {"query": "attr", "name": "age"}
• {"query": "path", "path": "[0]"} is equivalent to {"query": "item", "key": 0}
• {"query": "path", "path": "[key]"} is equivalent to {"query": "Item", "key": "key"}
• {"query": "path", "path": "bool()"} is equivalent to {"query": "bool"}
• {"query": "path", "path": "len()"} is equivalent to {"query": "len"}

Complex chain of selector queries can also be defined as a string:

• {"query": "path", "path": "children[0].age"} is equivalent to

{"query": "path", "path":
  [
    {"query": "attr", "name": "children"},
    {"query": "item", "key": 0},
    {"query": "attr", "name": "age"}
  ]
}

Variable Query

Since the queries are processed as a sequence, the only way to reuse a value in subsequent queries is to store it in a variable. This can be done using the var query {"query": "var", "name": "NAME", "value": "VALUE", "path": "PATH"}, which allows to keep the value accessible by unique NAME. The actual value to be assigned can be defined directly by VALUE or retrieved from the input data using PATH selector (see path). If both VALUE and PATH remain undefined, the value is assigned directly by the input data. The query itself passes the input data unchanged, so it can be used in the middle of a sequence without affecting the output of previous queries. Assigned value can be accessed by its name in attr query or anywhere the PATH is expected.

[
    {"query": "var", "name": "parent_age", "path": "age"},
    {"query": "attr", "name": "children"},
    {"query": "item", "key": 0},
    {"query": "<", "left": "age", "right": "parent_age"}
]

Projection Queries

On collections of items, several projection queries can be applied to retrieve specific value from each item. The PATH selector (see path) is applied to each item in the input data and the output is collected together in a new list.

• Select: {"query": "select", "path": "PATH"} is equivalent to [PATH(x) for x in input_data]
• Many: {"query": "many", "path": "PATH"} is equivalent to [y for x in input_data for y in PATH(x)]

[
    {"query": "many", "path": "children"},
    {"query": "select", "path": "name"}
]

Aggregation Queries

Several aggregation queries are available to retrieve a single value from the input data. The PATH selector (see path) is applied to each item in the input data and the output is aggregated together by specific operation.

• Count: {"query": "count", "path": "PATH"} equivalent to sum(1 for x in input_data if PATH(x))
• Sum: {"query": "sum", "path": "PATH"} is equivalent to sum(PATH(x) for x in input_data)
• Mean: {"query": "mean", "path": "PATH"} is equivalent to sum(PATH(x) for x in input_data) / len(input_data)

[
    {"query": "sum", "path": [
        {"query": "attr", "name": "children"},
        {"query": "len"}
    ]}
]

Partitioning Queries

Portion of the input data can be retrieved using simple partitioning queries.

• Take: {"query": "take", "count": "COUNT"} is equivalent to input_data[:COUNT]
• Skip: {"query": "skip", "count": "COUNT"} is equivalent to input_data[COUNT:]
• Slice: {"query": "slice", "start": "START", "end": "END", "step": "STEP"} is equivalent to input_data[START:END:STEP]

Search Queries

Several conditional queries are available to retrieve specific items, where the PATH selector (see path) is applied to each item and only those evaluated to True are included in the final selection. Some of those queries may raise custom IterationError exception if the expected number of items is not found.

• Where: {"query": "where", "path": "PATH"} is equivalent to filter(PATH(x) for x in input_data)
• First: {"query": "first", "path": "PATH"} is equivalent to next(x for x in input_data if PATH(x))
• Last: {"query": "last", "path": "PATH"} is equivalent to next(x for x in reversed(input_data) if PATH(x))
• Single: {"query": "single", "path": "PATH"}
• Any: {"query": "any", "path": "PATH"} is equivalent to any(PATH(x) for x in input_data)
• All: {"query": "all", "path": "PATH"} is equivalent to all(PATH(x) for x in input_data)
• Noone: {"query": "none", "path": "PATH"}
• Distinct: {"query": "distinct", "path": "PATH"}
• Min: {"query": "min", "path": "PATH"} equivalent to min(input_data, key=PATH)
• Max: {"query": "max", "path": "PATH"} equivalent to max(input_data, key=PATH)

[
    {"query": "where", "path":
        {"query": "and", "queries": [
            {"query": ">", "left": "age", "value": 20},
            {"query": "not_empty", "left": "children"}
        ]}
    },
    {"query": "select", "path": "name"}
]

Sorting Query

The input data can be sorted using the sort query {"query": "sort", "path": "PATH", "reverse": "REVERSE"}, either by natural order or by a specific key defined by the PATH selector (see path). Optionally, the sorting can be reversed by setting REVERSE attribute to true. The sorting query returns a new sorted list.

{"query": "sort", "path": "age", "reverse": true}

Condition Queries

Essential functionality of the JqSON library is to evaluate conditions on the input data. This can be done by various conditional queries, comparing values retrieved from the input data or directly specified. To retrieve a left value from the input data, the LEFT must be defined as PATH selector (see path). Similarly, a right value can be retrieved by RIGHT or directly specified by VALUE.

• ==: {"query": "==", "left": "LEFT", "right": "RIGHT", "value": "VALUE"}is equivalent to LEFT == RIGHT

• !=: {"query": "!=", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to LEFT != RIGHT

• >: {"query": ">", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to LEFT > RIGHT

• <: {"query": "<", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to LEFT < RIGHT

• >=: {"query": ">=", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to LEFT >= RIGHT

• <=: {"query": "<=", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to LEFT <= RIGHT

• in: {"query": "in", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to LEFT in RIGHT

• contains: {"query": "contains", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to RIGHT in LEFT

• has: {"query": "has", "left": "LEFT", "right": "RIGHT", "value": "VALUE"} is equivalent to hasattr(LEFT, RIGHT)

• true: {"query": "true", "left": "LEFT"} is equivalent to bool(LEFT) == True

• false: {"query": "false", "left": "LEFT"} is equivalent to bool(LEFT) == False

• null: {"query": "null", "left": "LEFT"} is equivalent to LEFT is None

• not_null: {"query": "not_null", "left": "LEFT"} is equivalent to LEFT is not None

• empty: {"query": "empty", "left": "LEFT"} is equivalent to LEFT is None or LEFT == "" or LEFT == [] or LEFT == () or LEFT == {}

• not_empty: {"query": "not_empty", "left": "LEFT"} is equivalent to LEFT is not None and LEFT != "" and LEFT != [] and LEFT != () and LEFT != {}

{"query": "and", "queries": [
    {"query": ">", "left": "age", "value": 20},
    {"query": "not_empty", "left": "children"},
    {"query": ">", "left": "children.len()", "value": 1},
    {"query": ">", "left": "children[0].age", "right": "children[1].age"}
]}

Logical Queries

Multiple conditions can be combined using logical queries to behave as a single query. The QUERIES attribute is a used to define a list of individual queries, which are processed in order and their output is combined by specific logical operand. Note that unlike in the path query, the output of each individual query is not passed to the next one, but instead all queries are processed independently on the same input data.

• AND: {"query": "and", "queries": "QUERIES"} is equivalent to all(q(input_data) for q in QUERIES)
• OR: {"query": "or", "queries": "QUERIES"}, is equivalent to any(q(input_data) for q in QUERIES)
• NOT: {"query": "not", "queries": "QUERIES"}, is equivalent to not any(q(input_data) for q in QUERIES)

{"query": "and", "queries": [
    {"query": "<", "left": "age", "value": 20},
    {"query": ">", "left": "age", "value": 5}
]}

Disclaimer

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.