trino-sql-validator

Fast Trino SQL syntax validator — a Python library whose core is written in Rust and compiled to a native extension via PyO3 + maturin.

Installable from PyPI:

pip install trino-sql-validator

Quickstart

from trino_sql_validator import analyze_statements, validate, validate_file

# A string with one or many statements
result = validate("SELECT 1; SELECT * FROM t WHERE a > 0;")
assert result.valid
assert result.statement_count == 2

# Invalid SQL returns a value, never raises
result = validate("SELECT * FORM t")
assert not result.valid
print(result.error)          # e.g. "Expected: end of statement, found: FORM at line 1, column 10"
print(result.error.line)     # 1

# Validate a file
result = validate_file("queries.sql", dialect="trino")

# Opt-in per-statement metadata; indexes are zero-based
analysis = analyze_statements("EXPLAIN SELECT 1; CALL system.custom_proc()")
assert analysis.validation.valid
assert analysis.statements[0].kind == "explain"
assert analysis.statements[0].inner_kind == "query"
assert analysis.statements[1].kind == "call"

Invalid SQL (and files containing it) is returned as a ValidationResult; it is not raised as an exception. Only genuine misuse (unknown dialect, unreadable file) raises.

Catalog warnings (functions and data types)

For dialect="trino", validate() also checks that every function called and every data type used in the SQL exists in the documented Trino catalog. Unknown names are reported as non-fatal warnings — valid stays True because syntax is fine:

result = validate("SELECT marh(1.5)")       # round() misspelled
assert result.valid
print(result.warnings)                      # (FunctionWarning(name='marh', line=1, column=8),)
print(result.unknown_functions)             # ['marh']

result = validate("CREATE TABLE t (a bignum, b bigint)")  # bigint vs bignum
print(result.warnings[0])                   # TypeWarning(name='bignum', line=1, column=19)
print(result.unknown_types)                 # ['bignum']

The catalogs are auto-generated from the Trino docs and only check name existence, not argument counts, precision/scale, or semantic correctness. hive/generic dialects skip these checks. False positives are possible if a deployed Trino adds plugin functions/types beyond the docs.

Inline WITH FUNCTION names are exempt only within their own query scope. Qualified calls with the same final name are still checked. Procedure names in CALL and ALTER TABLE ... EXECUTE are not scalar functions and therefore do not produce FunctionWarning; their existence, parameters, arity, permissions, and connector availability require a Trino coordinator and are out of scope.

Statement metadata

analyze_statements() is an opt-in API that returns the unchanged ValidationResult together with a tuple of StatementInfo. Each entry contains a zero-based index, source span, source-derived kind, and (for EXPLAIN or PREPARE) an inner_kind when it can be identified. On invalid multi-statement input, error_statement_index identifies the source statement when the parser provided a location. Existing validate() and validate_file() return types are unchanged.

dbt and Jinja templates

Jinja/dbt SQL is supported by default. validate() and validate_file() use jinja="auto" to mask Jinja expressions, statements, and comments before parsing while preserving line numbers and file structure. This supports constructs such as {{ ref("orders") }} and {{ var("catalog") }} without requiring a dbt installation or project context. Use jinja="mask" as an explicit spelling of the same mode.

Use jinja="reject" to pass the original template directly to the SQL parser. Masking cannot determine SQL generated by control-flow blocks, macros, or adapter semantics; render those cases with dbt and validate the rendered SQL for complete coverage.

Dialects

• "trino" (default) — Trino-flavored with a custom override tuned for current Trino syntax, including Iceberg branches/time travel, complex nested types, routines, table functions, SQL/JSON, and Trino-specific DDL.
• "hive" and "generic" — offered as permissive alternates.

Known limitations

sqlparser-rs (the parser we use) performs syntax validation, not semantic analysis. It may accept SQL that Trino would reject at analysis time (unknown columns/tables, duplicate columns), and it can reject exotic Trino-specific DDL. The validator has targeted compatibility parsing for documented Trino syntax, including nested ROW/ARRAY/MAP types, but it does not replace Trino's semantic analyzer. See plan/roadmap.md for the path toward stricter Trino fidelity.

Parser fidelity is checked reproducibly against direct-string cases extracted from Apache Trino's parser tests. With ordinary Java strings and text blocks, the pinned Trino 483 audit currently accepts 476/484 statements, 231/238 expressions, 68/68 types, the extracted Functions/Routines subset, and rejects 23/23 direct negative statements. Known differences are pinned in a named allowlist; new mismatches or a reduced extracted denominator fail the audit. These figures and the 276 independently checked positive fixture statements describe measured corpora, not complete Trino grammar or connector behavior. SQL embedded inside ordinary string literals, JSON paths, WKT, dynamic SQL, and unrendered macro output is intentionally opaque rather than recursively parsed.

To keep invalid or adversarial input from exhausting the native parser stack, validation rejects a statement after 4,096 significant SQL tokens, nesting deeper than 256 groups or routine blocks, and an input after 65,536 significant tokens. The failure is returned as an ordinary invalid ValidationResult; when the limiting token has a source position, that position is included in the error. Semicolon-separated statements have independent per-statement budgets.

Development

See AGENTS.md for setup, internal conventions, and release steps. Key commands:

python3 -m venv .venv && source .venv/bin/activate
pip install -U pip maturin && pip install -e ".[dev]"
maturin develop          # build + install native ext into the venv
cargo test               # Rust tests
pytest -q                # Python tests
cargo fmt --check        # formatting
cargo clippy --all-targets -- -D warnings
python tools/extract_functions.py --ref 483 --check
python tools/extract_types.py --ref 483 --check
python tools/audit_upstream_parsers.py --baseline plan/trino_483_audit_baseline.json --fail-on-regression

License

MIT