datafusion-query-builder

A programmatic, injection-safe builder for DataFusion SQL — a typed Rust core with a pyo3-exposed Python API. It replaces hand-rolled f-string / template-literal SQL with a typed, composable surface, while still emitting SQL text (so the SQL stays visible in logs/traces, greppable, and cache-keyable).

Values are escaped by construction, so untrusted input (column values, user filters) is safe to embed. Strings, arrays, and numbers are quoted/encoded for you; an explicit raw(...) escape hatch is the only unescaped path.

Install

pip install datafusion-query-builder

Prebuilt abi3 wheels are published for CPython 3.9+ on macOS (Apple Silicon + Intel) and Linux (x86_64 + aarch64).

Quick start (Python)

from datafusion_query_builder import col, lit, param, raw, when, and_, table
from datafusion_query_builder import functions as f

q = (
    table("records")
    .filter((col("kind") == "span") & col("deployment_environment").is_in(["prod", "staging"]))
    .select(
        f.coalesce(col("service_name"), "(unknown)").alias("service"),
        f.approx_distinct(col("trace_id")).alias("request_count"),
        f.approx_percentile_cont(col("duration"), 0.95).alias("p95"),
    )
    .group_by(col("service_name"))
    .order_by(col("request_count").desc())
    .limit(200)
)
print(q.to_sql())

Bare Python scalars auto-promote to literals (so col("x") == "prod" works), and string/array literals are escaped — values are injection-safe by construction. Reach for raw(...) (a SQL fragment), f.call("name", ...) (any function), or param("name") (a ${name} placeholder) when you step outside the v1 grammar.

The JSONB key-exists operators are first-class: col("attributes").has_key("gen_ai.input.messages") renders attributes ? 'gen_ai.input.messages' — a cheap presence check that never extracts the value (contrast raw("attributes ->> 'k'").is_not_null(), which reads the whole value out just to test it). has_any_key and has_all_keys render ?| / ?&.

Architecture

façade types  ──lower.rs──▶  sqlparser::ast  ──Display──▶  SQL text
expr.rs / query.rs / functions.rs        (the only file that names sqlparser::ast)
  • expr.rs, query.rs, functions.rs — span-free, Default-friendly façade enums. Immutable / generative: every method returns a new value.
  • lower.rs — the single boundary to sqlparser::ast. A sqlparser version bump surfaces here and nowhere else.
  • render.rs — to_sql() plus validate() (renders then re-parses to prove well-formedness).
  • python.rs — Expr / Query wrappers, the f.* functions namespace, operator overloading with scalar→literal coercion. Gated behind the python feature.

sqlparser is pinned to the pydantic dollar-brace-0.62.0 fork via [patch.crates-io] — the same parser the DataFusion ecosystem uses, including the ${var} placeholder extension. The crate does not depend on DataFusion itself (only the test oracle does, optionally).

Develop & test

# Rust core (no Python toolchain needed):
cargo test --test core                       # rendering snapshots + regressions
cargo test --features datafusion-oracle --test properties   # property tests, see below
cargo clippy --all-targets --features datafusion-oracle -- -D warnings

# Coercion tests against a real embedded interpreter (needs PYO3_PYTHON -> a 3.9+ interpreter):
PYO3_PYTHON=$PWD/.venv/bin/python cargo test --lib --features test-embed

# Python extension:
uv venv && uvx maturin develop
python tests/test_python.py

How the tests are layered

The crate is correctness-critical (it generates SQL from user-controlled input), so the test surface is layered:

  • tests/properties.rs (proptest, behind datafusion-oracle) uses DataFusion itself as the oracle — it renders a query to SQL, then plans + executes it through real DataFusion and reads the value back. This upgrades "the SQL re-parses" to "the value survives a real engine round-trip", catching silent mis-encoding (escaping, float formatting, operator precedence). DataFusion is an optional, test-only dependency — never compiled into the lib or the wheel. tests/properties.proptest-regressions pins seeds that previously found bugs.
  • src/python.rs coercion_tests (behind test-embed) unit-test the Python-type → façade-literal boundary that can only be exercised with real Python objects (bool-before-int, big ints, non-finite floats, list/tuple). They compose with the property tests: coercion proves "Python value → correct value", the property tests prove "value → correct SQL → correct result".

When a property/coercion test finds a bug, fix it in the crate — that is the whole point of the library: one fix covers every caller.

Release

Wheels are built and published to PyPI by CI (.github/workflows/ci.yml) on a v* tag, using PyPI Trusted Publishing (OIDC) — no API tokens. To cut a release: bump version in Cargo.toml and pyproject.toml, tag vX.Y.Z, and push the tag.

License

MIT

Project links

Data verified by PyPI on Jul 19, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

Owner

PyPI data
Data sourced directly from PyPI's database.

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for adriangb from gravatar.com adriangb

Credits

Author: Pydantic

GitHub Statistics

Data verified by PyPI on Jul 19, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.9

Tags

datafusion sql query-builder sqlparser

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datafusion_query_builder-0.2.0.tar.gz (60.3 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ x86-64

datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.8 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64

datafusion_query_builder-0.2.0-cp39-abi3-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

File details

Details for the file datafusion_query_builder-0.2.0.tar.gz.

File metadata

  • Download URL: datafusion_query_builder-0.2.0.tar.gz
  • Upload date:
  • Size: 60.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for datafusion_query_builder-0.2.0.tar.gz
Algorithm Hash digest
SHA256 ad13c6948f115ea44b0fa52f2b272a3eba7fd377f6cd638ce5d461e9273dea0f
MD5 59e3929f8a41b2210e73aacd857ffa12
BLAKE2b-256 30c21ec0dde7f76e6ec60f07089ec2e4adafdae1ae0c678e008bef775f337fbd

See more details on using hashes here.

Provenance

The following attestation bundles were made for datafusion_query_builder-0.2.0.tar.gz:

Publisher: ci.yml on pydantic/datafusion-query-builder

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 945f848723e85cdbd5417b3297d345bd26e168a8c5828c39225fa460c4adf622
MD5 4b2dc6c7723e67c74ed95b8ad57f4227
BLAKE2b-256 8b2259830363eefbda7bff1ea74acb5c6e8d979c145770638da894b6a25f2858

See more details on using hashes here.

Provenance

The following attestation bundles were made for datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on pydantic/datafusion-query-builder

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 38704e39f0cb25bd2808064babc7f5e174c8e443bb4459619ba875079ce4a209
MD5 c8e0af776cb25f55f079abe466cac0e9
BLAKE2b-256 f95636c590e5f25def52963f3718ad977e46cb3c3af106d98e226b9ce0bd20e7

See more details on using hashes here.

Provenance

The following attestation bundles were made for datafusion_query_builder-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: ci.yml on pydantic/datafusion-query-builder

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datafusion_query_builder-0.2.0-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for datafusion_query_builder-0.2.0-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 def71e9c756aed93a0833bdf7a2c24298794865495d5af5beb57ceb3c26c9cb8
MD5 bbb6c6ad680e63fe2735645df7bc0848
BLAKE2b-256 2a86cb962bd4559349fc2291a3d6c793453ab9a956af139d27397dffd6b1cf77

See more details on using hashes here.

Provenance

The following attestation bundles were made for datafusion_query_builder-0.2.0-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: ci.yml on pydantic/datafusion-query-builder

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page