fast-pysqlparse: High-Performance SQL Parsing Library

Build Status Version Language License

README.md (Chinese)

A high-performance, cross-platform, lightweight SQL parsing library whose core trait is speed — built on a C++17 core with native Python bindings, it rapidly performs structured parsing of SQL, especially statements with highly complex structure and deep nesting.

Overview

fast-pysqlparse aims to overcome the performance and capability limits of traditional Python SQL parsers. By moving compute-intensive parsing into a native C++ layer, it maintains fast parsing even on large, deeply nested, structurally complex SQL.

The parser primarily parses ANSI-style SQL. For statements with a specific dialect, specifying the dialect is recommended; supported dialects: MySQL, PostgreSQL, SQLite, Doris.

Features

  • Fast SQL Parsing: Leverages a high-performance C++17 core to parse SQL statements rapidly
  • Structured Parsing: Oriented toward complex structure and deep nesting, especially statements mixing CTEs with SELECT/INSERT/VIEW
  • Cross-Platform: Compiled into native extensions (.pyd for Windows, .so for Linux)
  • Comprehensive SQL Support: Supports a wide range of SQL statements, including:
    • SELECT (with complex JOIN, WHERE, GROUP BY, subqueries, etc.)
    • INSERT
    • Data Definition Language (CREATE)
    • VIEW
    • DELETE
    • UPDATE
    • Common Table Expressions (CTEs), including nested CTEs
  • Abstract Syntax Tree (AST): Generates a detailed JSON representation of the parsed SQL AST for easy traversal and analysis
  • SQL Formatting: Automatically reformats messy SQL into a clean, readable structure
  • Table Lineage Parsing: Automatically traces and reveals the source-to-target relationships between tables (data lineage)
  • Tokenization: Breaks down SQL statements into their fundamental tokens for lexical analysis
  • Python API: A clean and intuitive Python library built around the high-speed native extension

Performance

This library is engineered for speed. By moving the computationally intensive parsing work to a native C++ layer, it significantly outperforms pure-Python parsing libraries, especially when dealing with large, complex SQL scripts.

Benchmark Results

Test 1: 5000 Iterations

  • SQL Length: 639 characters
  • Total Time: 0.48s
  • PPS (Parses Per Second): ~10300
  • Average per parse: ~0.097ms

Test 2: 10 Million Character SQL

  • SQL Length: 10,500,998 characters
  • Total Time: 0.54s
  • CPS (Characters Per Second): ~19,500,000
  • Parse successful!

Test 3: Python-Only Comparison on ~10M PostgreSQL SQL (No C Benchmark)

Benchmark script: test/python_parsers_10m.py

Parser Avg Time CPS
fastsqlparse 0.7394s 13,524,892.44
pglast 4.8541s 2,060,217.57
sqlglot (postgres) 20.5364s 486,963.12
sqlparse 84.9591s 117,709.35

Notes:

  • SQL size: 10,000,484 chars
  • Runs per parser: 1
  • Results source: results of test/python_parsers_10m.py

Installation

pip install fast-pysqlparse

From Source:

git clone https://github.com/Nohaltsail/fast-pysqlparse.git
cd fast-pysqlparse
pip install build
python -m build
cd dist
pip install fast_pysqlparse-*.whl

Quick Start

from fastsqlparse import Parsed, ParsedQuery

if __name__ == '__main__':
    sql = """

-- main query
SELECT 
    'Monthly Sales Report' AS report_type,
    ms.year,
    ms.month,
    ms.region,
    ms.customer_segment,
    ms.unique_customers,
    ms.total_orders,
    ms.gross_sales,
    ms.avg_order_value,
    ms.cancelled_orders,
    (SELECT SUM(gross_sales) FROM sub_monthly_sales WHERE year = ms.year AND month = ms.month) AS total_monthly_sales,
    ms.gross_sales / NULLIF((SELECT SUM(gross_sales) FROM monthly_sales WHERE year = ms.year AND month = ms.month), 0) * 100 AS sales_percentage,
    (SELECT AVG(avg_order_value) FROM monthly_sales WHERE year = ms.year AND month = ms.month) AS overall_avg_order_value
FROM monthly_sales ms

UNION ALL

SELECT 
    'Category Performance' AS report_type,
    cs.year,
    cs.month,
    NULL AS region,
    cs.category_name AS customer_segment,
    cs.unique_buyers AS unique_customers,
    cs.order_count AS total_orders,
    cs.total_sales AS gross_sales,
    cs.total_sales / NULLIF(cs.order_count, 0) AS avg_order_value,
    NULL AS cancelled_orders,
    (SELECT SUM(total_sales) FROM sub_category_sales WHERE year = cs.year AND month = cs.month) AS total_monthly_sales,
    cs.total_sales / NULLIF((SELECT SUM(total_sales) FROM category_sales WHERE year = cs.year AND month = cs.month), 0) * 100 AS sales_percentage,
    NULL AS overall_avg_order_value
FROM category_sales cs
LIMIT 50, 100"""

    sql_len = len(sql)
    print("sql length: ", sql_len)

    # parse sql statements to SQL object
    sql_stmt = Parsed(sql)
    # Format and print the SQL statement with proper indentation
    print(sql_stmt.format())  # Output formatted SQL statement

    # Tokenization - returns list of tuples containing (token_value, token_type, position)
    tokens = ParsedQuery.tokenize(sql)  # Get tuple list of token information (token_value, token_type, position)

    # Alternative tokenization - returns list of token objects with attributes
    token_obj_list = sql_stmt.tokens()  # Get object list of token information

    # Generate and print Abstract Syntax Tree (AST) in JSON format
    print(sql_stmt.AST())  # Get JSON structure of the SQL statement

    # Extract table lineage/dependencies from the query
    src_tables = ParsedQuery.parse_dependence(sql)  # Get source tables (dependencies) of the query

Comment Handling (pure)

pure controls SQL comment handling in parser constructors such as Parsed, ParsedQuery, ParsedInsert, ParsedCTE, ParsedUpdate, ParsedDelete, ParsedView, and ParsedCreate.

  • pure=False (default): keep comments in parsing/formatting output.
  • pure=True: strip -- and /* ... */ comments before parsing; formatted output and token results exclude comments, and parsing may be faster.
from fastsqlparse import Parsed

parsed_keep = Parsed(sql, pure=False)  # preserve comments
parsed_clean = Parsed(sql, pure=True)  # strip comments before parse

Dialects (dialect)

dialect selects the SQL dialect for parsing and lexical analysis (default "ansi"). Every parser constructor accepts dialect as a string; tokenize classmethods and ParsedQuery.parse_dependence accept a DialectType (default DialectType.ANSI).

Supported dialects: ansi, mysql, postgresql, sqlite, doris (see the Dialects enum and the DIALECT_* constants in fastsqlparse.conf).

from fastsqlparse import Parsed, ParsedQuery, Dialects, DialectType

parsed = Parsed(sql, dialect=Dialects.MYSQL.value)          # "mysql"
query = ParsedQuery(sql, "q", dialect="postgresql")
ParsedQuery.tokenize(sql, dialect=DialectType.MYSQL)        # typed DialectType
ParsedQuery.parse_dependence(sql, dialect="mysql")

When to Use Which Parser

Scenario Parser to Use
SQL statement type is unknown or you don't want to specify the type Parsed/ParsedOne
Multiple SQL statements separated by ; (script execution) Parsed
SELECT / query statement ParsedQuery
INSERT statement ParsedInsert
DELETE statement ParsedDelete
UPDATE statement ParsedUpdate
CREATE TABLE statement ParsedCreate
CREATE VIEW statement ParsedView
CTE (WITH clause) statement ParsedCTE

Note: If your SQL contains multiple statements separated by semicolons (e.g., a script with CREATE, INSERT, SELECT), you must use Parsed. The type-specific parsers are designed for single, known-type statements only.

Documentation

For complete API documentation, see: API_DOC.md

Contributing

Contributions are welcome! Please feel free to submit pull requests, report bugs, or suggest new features.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Note on Dynamic Libraries

This project currently distributes precompiled dynamic libraries (.pyd and .so). The corresponding C++ source code for these dynamic libraries is temporarily not public and is planned to be opened in a future release.

For the full supplementary notice, see LICENSE.

You can also use the dynamic libraries from the source code directly to develop your own SQL parsing library.

Project links

Data verified by PyPI on Aug 28, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.3)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for Nohaltsail from gravatar.com Nohaltsail

Credits

Author: NohaltSial

GitHub Statistics

Data verified by PyPI on Aug 28, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.10

Tags

sql parser ast fast

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

fast_pysqlparse-0.8.1-cp314-cp314-win_amd64.whl (1.6 MB view details)

Uploaded CPython 3.14Windows x86-64

fast_pysqlparse-0.8.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.1-cp313-cp313-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.13Windows x86-64

fast_pysqlparse-0.8.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.1-cp312-cp312-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.12Windows x86-64

fast_pysqlparse-0.8.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.1-cp311-cp311-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.11Windows x86-64

fast_pysqlparse-0.8.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.1-cp310-cp310-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.10Windows x86-64

fast_pysqlparse-0.8.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

File details

Details for the file fast_pysqlparse-0.8.1-cp314-cp314-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 1d8d7dd499a2bc25ce7539c7614e2a25e313593e724d9ae7913174fb1cb68e84
MD5 43b167e57f67709edd8fd167c67c462d
BLAKE2b-256 f76753ef14fdfc26686189c858c29eaec07afe95a349a60b34910bb8abc6eb1d

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp314-cp314-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 97d62b0e98a9d150edab48412794eb586306762a1a6efe6d80200a6542d41fee
MD5 a64ba8bdfee1e84b772612fb76a2e837
BLAKE2b-256 d35b13cd9f42cb203c49e7cdb3f532eaf5b15a52d6e83c5124345087b8504ad2

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp313-cp313-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 cc1041d18fb7fe5751c8408e2be1c68113b6f827277cb52ecc79a8e46098d1aa
MD5 5b9996f79d175146fdc689694a3d8ef3
BLAKE2b-256 7bbc47e086a190550b4a842e6a71f902d03fcee06cc958ef02c69981062ed8e4

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp313-cp313-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 1f31d6a7263c471978e7b646ad9a47b5e8b2784856fda4bbe56a71cad37aefbe
MD5 f4f8cb34ff43687559639abb1deacc2c
BLAKE2b-256 baee01e3043f0a1838dd2db36ba56714a277ccd310d8827c0e188af96a40a3cd

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 7e133e53d928906b2e7182af70e94316ce53297301b6bc3b9f7cbe84216af7fa
MD5 bc2b1868f09179975fdefd5a330d9f6e
BLAKE2b-256 ca89b753fc7eb350de30dc880dcc8539963cdebc3ac7c70a299678e2a90cd2a5

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp312-cp312-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 fdf4ba036f6be61cbd4b59cad7ae8140f6fc085482488225a8ec67a3adbfc812
MD5 a838df2a0d1eb8fd20bf4e9b58820fff
BLAKE2b-256 fd395b132c6d8c76c9dc9ed8b3232f90bd3d32ab840c3339fd11a4dc7dc3cef1

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 640abb13adc04114c669f8e34079097641bdd4f3a3c705d6646b710406223c27
MD5 c1628f3b8e4e01667811babd14ebdf78
BLAKE2b-256 436ca06a5268734f1f29dfe28af20c10e7bb673c349e3efa59939a3ff7b68bb0

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp311-cp311-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 c9fdde6c376162a1ff800813b179be2428b3cbaf399cc451573564617d536e6f
MD5 9a05d639431fa8d1d333bc61c73625e5
BLAKE2b-256 78c7abe46d70295fd4fbdc1aa498c43ccbd8706c9176e6639a207bb197466dde

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 d3801f4a8d1d00b210f99007f7ff9df588d2a90bb8f4b6ac28bd5acfb72b250f
MD5 b0cc90f3fb78475c529ccc2ed35b2066
BLAKE2b-256 125e2615772430207c1571837f10b2d51be46d4e7eee89fe4ff8e35b0bf1d872

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp310-cp310-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 01a10134cac8dce8d4b8d265c45550a76c82f1d6696424109d7f8ccffb3535b1
MD5 0c8e49253a64c421bb88183d186f782c
BLAKE2b-256 6d79d7ea872954495965d8309221a2d6259193a030ed61d68274a93fee16414e

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.8.3

10 files

0.8.2

10 files

This release

0.8.1 This release

10 files

0.8.0

10 files

0.7.1

10 files

0.7.0

10 files

0.6.1

10 files

0.6.0

2 files

0.5.1

2 files

0.5.0

2 files

0.4.0

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.9

2 files

0.2.8

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.9

2 files

0.1.8

2 files

0.1.7

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page