aero-jmespath

A native JMESPath kernel written in Aero, compiled to a Python C-extension. It parses JSON, evaluates a JMESPath expression, and serializes the result entirely in native code — no Python objects are built in between.

What it is

jmespath.py is a pure-Python implementation. Its standard usage is:

import jmespath
result = jmespath.search("servers[?status == `active`].name", data_dict)

aero-jmespath provides a drop-in equivalent that takes the raw JSON string and returns a JSON string, skipping the Python-object round-trip:

import kernel
result = kernel.search("servers[?status == `active`].name", json_str)

The Aero kernel runs the full pipeline — JSON parse, expression parse, evaluation and serialization — in one native pass over the input.

Build

Requires the Aero toolchain (64-bit Windows / macOS / Linux). From the repo root:

aero build src/kernel.aero --pyext     # produces src/kernel.pyd / kernel.so

Usage

import sys
sys.path.insert(0, "src")
import kernel

kernel.search("@", '{"a": 1}')                       # '{"a":1}'
kernel.search("a.b", '{"a": {"b": "x"}}')            # '"x"'
kernel.search("servers[*].name", open("d.json").read())

search(expr, json_str) returns the serialized result as a JSON string, or "\x1eERR1" (syntax error) / "\x1eERR2" (invalid-value error).

Compile-once API

Repeated queries on the same expression skip the lexer/parser entirely: compile the expression once to an opaque blob, then evaluate the blob against any number of documents. This is what aero_jmespath.search() uses internally (with a bounded in-process cache), and it is also exposed directly:

import aero_jmespath

blob = aero_jmespath.compile("servers[?status == `active`].name")   # once
result = aero_jmespath.search_cached(blob, json_str)                 # per document

Batch queries on one document

multi() parses the JSON document exactly once and evaluates many compiled expressions against the same parsed document in native code — the batch workload where jmespath.py reuses a pre-parsed dict:

import aero_jmespath

results = aero_jmespath.multi(
    ["servers[*].name",
     "servers[?status == `active`].instances[0].id",
     "servers[0].instances[*].cpu"],
    json_str,
)   # list of JSON result strings, one per expression

compile raises ValueError on a bad expression; multi raises ValueError on a bad expression or a non-JSON document.

Compliance

Passes all 578 official jmespath compliance tests (identifiers, indices, slices, filters, projections, multiselect, literals, escapes, wildcards, booleans).

tests/run_compliance.py

basic.json       ok=18   fail=0
boolean.json     ok=60   fail=0
...
TOTAL ok=578 fail=0

Integration with jmespath.py

aero-jmespath is designed to be the optional native accelerator for jmespath.py. Once installed (pip install aero-jmespath), jmespath.search_json(expr, json_string) runs the whole parse + eval + serialize pipeline in native code and falls back to pure Python when the package is absent, so behaviour never changes:

import jmespath
result = jmespath.search_json("servers[?status == `active`].name", json_str)

aero_jmespath.search(expression, json_string) itself returns a JSON string, so it can also be used directly.

Performance

For one-shot queries — parse + eval + serialize per call, the typical CLI / stream / per-request workload where the input is a JSON string — the native kernel is 1.6–2.2× faster than jmespath.py (json.loads + compiled search), and the advantage grows with document size:

document	aero-jmespath	jmespath.py	speedup
0.71 MB	5.6 ms	9.1 ms	1.63×
3.57 MB	30.2 ms	56.9 ms	1.88×
7.19 MB	61.7 ms	133.8 ms	2.17×

Query: servers[?status == active].instances[0].id on a synthetic server fleet. See bench/ for the scripts.

In a batch workload (many queries on a document that is already parsed once, e.g. compiled.search(dict) on a reused dict), a single search/search_cached call is slower because it re-parses the JSON string every time. But the native kernel has its own batch primitive that wins: multi() parses the document once and evaluates every expression against that same parsed arena, avoiding the pure-Python object walk that jmespath.py pays per expression:

8 expressions on one 0.71 MB doc (per batch of 8):
  aero_multi   5.98 ms/batch
  jmespath    14.34 ms/batch
  ratio        2.40x

The one-shot speedup (fresh parse per call) remains the string-input workload; multi() is the answer when one document drives many queries.

JSON parsing alone beats CPython's json.loads and the Rust-based orjson (on a 7.19 MB / 20000-server document, bench/parse.py):

aero parse_only  41.00 ms   (175 MB/s)
json.loads       82.40 ms   ( 87 MB/s)   -> 2.01x slower
orjson.loads     64.63 ms   (111 MB/s)   -> 1.58x slower

Expression-level comparison on a 0.71 MB / 2000-server document (queries/sec, higher is better; jp+str = jmespath including result json.dumps):

expression             aero (q/s)   jp+str (q/s)   vs jp+str
------------------------------------------------------------
simple field                  153           153      1.00x
wildcard projection           132           130      1.01x
filter + projection           121            96      1.27x
nested projection             111           113      0.98x
multiselect hash               89            87      1.02x
comparison filter             107            98      1.10x
slice                         102           152      0.67x

Architecture

• Flat value arena — every JSON value lives in one Vec<Jv>; arrays and objects store value indices, objects keep [key_index, value_index, ...] pairs so key names survive serialization.
• Fixed-point numbers — Aero has no i64→f64 conversion and extern "C" cannot return f64, so numbers are stored as i64 in micro-units (value × 1e6); all compliance numbers are short decimals well within range.
• Index-based serializer — serialization recurses over value indices rather than passing the 32-byte enum by value, so flat arrays of 1M+ elements serialize without exhausting the ~1 MB Python thread stack.
• Expression layer — a Pratt parser mirrors jmespath.py's binding-power table, producing an AST arena that the tree-walking evaluator consumes.

Files

src/kernel.aero          the Aero kernel (parser + evaluator + serializer)
tests/run_compliance.py  runs the official compliance suite
tests/test_parse.py      JSON parser smoke tests
tests/test_compile.py    compile() + search_cached() round-trip vs search()
tests/test_multi.py      multi() batch results vs per-expression search()
tests/test_native.py     integration test: search_json() accelerated path vs pure Python
bench/bench.py           expression-level benchmark vs jmespath.py
bench/batch.py           batch vs one-shot benchmark
bench/scale.py           speedup vs document size
bench/parse.py           JSON parse-only vs json.loads / orjson

The compiled extension is built locally (aero build src/kernel.aero --pyext) and is not committed to the repository.

License

MIT