trino-sql-validator

Fast Trino SQL syntax validator — a Python library whose core is written in Rust and compiled to a native extension via PyO3 + maturin.

Installable from PyPI:

pip install trino-sql-validator

Quickstart

from trino_sql_validator import analyze_statements, validate, validate_file

# A string with one or many statements
result = validate("SELECT 1; SELECT * FROM t WHERE a > 0;")
assert result.valid
assert result.statement_count == 2

# Invalid SQL returns a value, never raises
result = validate("SELECT * FORM t")
assert not result.valid
print(result.error)          # e.g. "Expected: end of statement, found: FORM at line 1, column 10"
print(result.error.line)     # 1

# Validate a file
result = validate_file("queries.sql", dialect="trino")

# Opt-in per-statement metadata; indexes are zero-based
analysis = analyze_statements("EXPLAIN SELECT 1; CALL system.custom_proc()")
assert analysis.validation.valid
assert analysis.statements[0].kind == "explain"
assert analysis.statements[0].inner_kind == "query"
assert analysis.statements[1].kind == "call"

Invalid SQL (and files containing it) is returned as a ValidationResult; it is not raised as an exception. Only genuine misuse (unknown dialect, unreadable file) raises.

Advisory warnings

For dialect="trino", validate() also checks that every function called and every data type used in the SQL exists in the documented Trino catalog. Unknown names are reported as non-fatal warnings — valid stays True because syntax is fine:

result = validate("SELECT marh(1.5)")       # round() misspelled
assert result.valid
print(result.warnings)                      # (FunctionWarning(name='marh', line=1, column=8),)
print(result.unknown_functions)             # ['marh']

result = validate("CREATE TABLE t (a bignum, b bigint)")  # bigint vs bignum
print(result.warnings[0])                   # TypeWarning(name='bignum', line=1, column=19)
print(result.unknown_types)                 # ['bignum']

result = validate("SELECT sum() OVER ()")
assert not result.valid
print(result.function_argument_warnings[0]) # sum expects 1 argument; got 0

result = validate("SELECT min(), max(1, 2, 3)")
assert not result.valid
assert len(result.function_argument_warnings) == 2

# A single call returns the first error AND all diagnostics
result = validate("SELECT sum(); SELECT min()")
assert not result.valid
print(result.error)                         # first error: sum()
print(result.function_argument_warnings)    # both sum() and min()

# Opt into the advisory behavior from 0.22.0 if needed
assert validate("SELECT sum()", function_arguments="warn").valid

# Preserve the syntax-only behavior from versions through 0.21.x
assert validate("SELECT sum()", function_arguments="off").valid

The contextual words ALL, OVER, PARTITION, RETURN, and AT are valid non-reserved Trino identifiers, but are easy to confuse with surrounding SQL syntax. Using one as an alias therefore produces a non-fatal AliasWarning; quote the alias to make the identifier intent explicit and suppress the warning:

result = validate("SELECT orderdate AS At")
assert result.valid
print(result.warnings[0])                   # ambiguous unquoted alias 'at' at line 1, column 21
print(result.ambiguous_aliases)             # ['at']

assert validate('SELECT orderdate AS "At"').warnings == ()

AT is also used by the temporal operators AT TIME ZONE and AT LOCAL. Those operator forms do not produce alias warnings.

Trino's 83 reserved keywords are stricter: an unquoted reserved alias is a syntax error, including after an explicit AS. Double quotes turn the word into a valid delimited identifier in aliases, object names, and column references; single quotes do not:

assert not validate("SELECT 1 AS where").valid
assert validate('SELECT 1 AS "where"').valid

assert validate('CREATE TABLE dwh_team."FROM" AS SELECT 1 AS "ALTER"').valid
assert validate(
    'SELECT "FROM"."ALTER" FROM dwh_team."FROM" AS "FROM"'
).valid

The catalogs are auto-generated from the Trino docs and check name existence. Version 0.22.0 additionally checks documented argument counts for 45 unqualified built-in aggregate functions, including sum, avg, min, max, min_by, max_by, statistical aggregates, map aggregates, and digest aggregates. Since 0.23.0, ordinary validate(sql) defaults to function_arguments="error": a mismatch makes the result invalid and preserves all diagnostics in warnings. error reports the first argument-count error; function_argument_warnings lists all of them. "warn" and "off" remain explicit options for advisory and syntax-only behavior. This is a curated rule rather than general overload resolution: argument types, functions outside the registry, precision/scale, and semantic correctness still require Trino. hive/generic dialects skip these checks. False positives are possible if a deployed Trino adds plugin functions/types beyond the docs.

Unknown names and ambiguous aliases remain advisory on their own. When parsing fails, no complete AST is available and warnings is empty. For compatibility, any invalid result has statement_count=0; analyze_statements() still exposes the source-derived statement list and the first error's statement index.

For catalog name checking, inline WITH FUNCTION names are exempt only within their own query scope, while qualified calls with the same final name are still checked. The argument-count rule is narrower and skips qualified calls because it does not perform name resolution. Procedure names in CALL and ALTER TABLE ... EXECUTE are not scalar functions and therefore do not produce FunctionWarning; their existence, parameters, arity, permissions, and connector availability require a Trino coordinator and are out of scope.

Statement metadata

analyze_statements() is an opt-in API that returns the unchanged ValidationResult together with a tuple of StatementInfo. Each entry contains a zero-based index, source span, source-derived kind, and (for EXPLAIN or PREPARE) an inner_kind when it can be identified. On invalid multi-statement input, error_statement_index identifies the source statement when the parser provided a location. Existing validate() and validate_file() return types are unchanged.

dbt and Jinja templates

Jinja/dbt SQL is supported by default. validate() and validate_file() use jinja="auto" to mask Jinja expressions, statements, and comments before parsing while preserving line numbers and file structure. This supports constructs such as {{ ref("orders") }} and {{ var("catalog") }} without requiring a dbt installation or project context. Use jinja="mask" as an explicit spelling of the same mode.

Use jinja="reject" to pass the original template directly to the SQL parser. Masking cannot determine SQL generated by control-flow blocks, macros, or adapter semantics; render those cases with dbt and validate the rendered SQL for complete coverage. If masking changes the input, function argument-count diagnostics are disabled to avoid inferring arity from an unrendered template.

Dialects

• "trino" (default) — Trino-flavored with a custom override tuned for current Trino syntax, including Iceberg branches/time travel, complex nested types, routines, table functions, SQL/JSON, and Trino-specific DDL.
• "hive" and "generic" — offered as permissive alternates.

Known limitations

sqlparser-rs (the parser we use) performs syntax validation, not semantic analysis. It may accept SQL that Trino would reject at analysis time (unknown columns/tables, duplicate columns), and it can reject exotic Trino-specific DDL. The validator has targeted compatibility parsing for documented Trino syntax, including nested ROW/ARRAY/MAP types, but it does not replace Trino's semantic analyzer. See plan/roadmap.md for the path toward stricter Trino fidelity.

Parser fidelity is checked reproducibly against direct-string cases extracted from Apache Trino's parser tests. With ordinary Java strings and text blocks, the pinned Trino 483 audit currently accepts 484/484 statements, 238/238 expressions, 68/68 types, the extracted Functions/Routines subset, and rejects 23/23 direct negative statements. The remaining error-suite differences are documented semantic or wrapper-boundary exceptions. Exact source and corpus hashes make changed, missing, or stale baseline cases fail the audit. These figures and the 553 independently checked positive fixture statements describe measured corpora, not complete Trino grammar or connector behavior. SQL embedded inside ordinary string literals, JSON paths, WKT, dynamic SQL, and unrendered macro output is intentionally opaque rather than recursively parsed.

An optional offline differential audit compares the same fixtures and Trino 483 corpus with pinned SQLGlot 30.19.0. SQLGlot is neither a runtime dependency nor an alternative validity backend; its full-AST, Command fallback, and error outcomes are tracked separately to identify candidates for native improvements.

The native Trino SqlParser oracle and grammar inventory are development tools, not parser backends. Importing or using the package never starts Java, Maven, SQLGlot, Docker, or a subprocess. The reproducible oracle uses JDK 25 and records the input, preparation, Java runtime, harness, class, and dependency-JAR hashes for every run. Its current corpus contains 1,040 independently labelled inputs. The grammar inventory fingerprints Trino's grammar, SqlParser, embedded PostProcessor, and AstBuilder; it records all 144 parser rules and 708 top-level alternatives without claiming complete branch coverage. The same Rust core can also be compiled as an rlib without PyO3 for unit and fuzz testing.

To keep invalid or adversarial input from exhausting the native parser stack, validation rejects a statement after 4,096 significant SQL tokens, nesting deeper than 256 groups or routine blocks, and an input after 65,536 significant tokens. The failure is returned as an ordinary invalid ValidationResult; when the limiting token has a source position, that position is included in the error. Semicolon-separated statements have independent per-statement budgets.

Development

See AGENTS.md for setup, internal conventions, and release steps. The current automated suite contains 5,201 pytest cases and 91 Rust unit tests. It includes a 28-case LF/CRLF transformation matrix, six native-labelled token deletion mutations, and 18 committed seeds for three bounded fuzz targets. Key commands:

python3 -m venv .venv && source .venv/bin/activate
pip install -U pip maturin && pip install -e ".[dev]"
maturin develop          # build + install native ext into the venv
cargo test               # Rust tests
pytest -q                # Python tests
cargo fmt --check        # formatting
cargo clippy --all-targets -- -D warnings
# optional differential parser audit
pip install -e ".[sqlglot-audit]"
python tools/audit_sqlglot.py --baseline \
  plan/sqlglot_30_19_0_trino_483_audit_baseline.json --fail-on-regression
python tools/extract_functions.py --ref 483 --check
python tools/extract_types.py --ref 483 --check
python tools/audit_upstream_parsers.py --baseline plan/trino_483_audit_baseline.json --fail-on-regression

License

MIT

Comparison with SQLGlot

trino-sql-validator and SQLGlot solve different problems. This library focuses on strict, fast validation of Trino syntax and returns a small validation result with Trino-specific catalog warnings. SQLGlot provides a rich AST and is a better fit for formatting, rewriting, lineage, and translation between SQL dialects, but its parser is deliberately permissive and is not a strict Trino validity oracle.

The reproducible audit below compares trino-sql-validator 0.22.0 with SQLGlot 30.19.0 against project fixtures and parser cases extracted from Trino 483. “SQLGlot accepted” includes its Command fallback, which preserves an unsupported statement as text without fully parsing it. “Full AST” excludes that fallback.

The measured corpora are regression benchmarks, not proof of complete Trino grammar coverage. SQLGlot remains an optional offline audit dependency and is not used by validate() at runtime. Confirmed SQLGlot findings are checked against the native Trino parser before being implemented in the Rust parser. See the full SQLGlot audit for methodology, known mismatches, and detailed results.