fast-pysqlparse: High-Performance SQL Parsing Library

Build Status Version Language License

README.md (Chinese)

A high-performance, cross-platform, lightweight SQL parsing library whose core trait is speed — built on a C++17 core with native Python bindings, it rapidly performs structured parsing of SQL, especially statements with highly complex structure and deep nesting.

Overview

fast-pysqlparse aims to overcome the performance and capability limits of traditional Python SQL parsers. By moving compute-intensive parsing into a native C++ layer, it maintains fast parsing even on large, deeply nested, structurally complex SQL.

The parser primarily parses ANSI-style SQL. For statements with a specific dialect, specifying the dialect is recommended; supported dialects: MySQL, PostgreSQL, SQLite, Doris.

Features

  • Fast SQL Parsing: Leverages a high-performance C++17 core to parse SQL statements rapidly
  • Structured Parsing: Oriented toward complex structure and deep nesting, especially statements mixing CTEs with SELECT/INSERT/VIEW
  • Cross-Platform: Compiled into native extensions (.pyd for Windows, .so for Linux)
  • Comprehensive SQL Support: Supports a wide range of SQL statements, including:
    • SELECT (with complex JOIN, WHERE, GROUP BY, subqueries, etc.)
    • INSERT
    • Data Definition Language (CREATE)
    • VIEW
    • DELETE
    • UPDATE
    • Common Table Expressions (CTEs), including nested CTEs
  • Abstract Syntax Tree (AST): Generates a detailed JSON representation of the parsed SQL AST for easy traversal and analysis
  • SQL Formatting: Automatically reformats messy SQL into a clean, readable structure
  • Table Lineage Parsing: Automatically traces and reveals the source-to-target relationships between tables (data lineage)
  • Tokenization: Breaks down SQL statements into their fundamental tokens for lexical analysis
  • Python API: A clean and intuitive Python library built around the high-speed native extension

Performance

This library is engineered for speed. By moving the computationally intensive parsing work to a native C++ layer, it significantly outperforms pure-Python parsing libraries, especially when dealing with large, complex SQL scripts.

Benchmark Results

Test 1: 5000 Iterations

  • SQL Length: 639 characters
  • Total Time: 0.48s
  • PPS (Parses Per Second): ~10300
  • Average per parse: ~0.097ms

Test 2: 10 Million Character SQL

  • SQL Length: 10,500,998 characters
  • Total Time: 0.54s
  • CPS (Characters Per Second): ~19,500,000
  • Parse successful!

Test 3: Python-Only Comparison on ~10M PostgreSQL SQL (No C Benchmark)

Benchmark script: test/python_parsers_10m.py

Parser Avg Time CPS
fastsqlparse 0.7394s 13,524,892.44
pglast 4.8541s 2,060,217.57
sqlglot (postgres) 20.5364s 486,963.12
sqlparse 84.9591s 117,709.35

Notes:

  • SQL size: 10,000,484 chars
  • Runs per parser: 1
  • Results source: results of test/python_parsers_10m.py

Installation

pip install fast-pysqlparse

From Source:

git clone https://github.com/Nohaltsail/fast-pysqlparse.git
cd fast-pysqlparse
pip install build
python -m build
cd dist
pip install fast_pysqlparse-*.whl

Quick Start

from fastsqlparse import Parsed, ParsedQuery

if __name__ == '__main__':
    sql = """

-- main query
SELECT 
    'Monthly Sales Report' AS report_type,
    ms.year,
    ms.month,
    ms.region,
    ms.customer_segment,
    ms.unique_customers,
    ms.total_orders,
    ms.gross_sales,
    ms.avg_order_value,
    ms.cancelled_orders,
    (SELECT SUM(gross_sales) FROM sub_monthly_sales WHERE year = ms.year AND month = ms.month) AS total_monthly_sales,
    ms.gross_sales / NULLIF((SELECT SUM(gross_sales) FROM monthly_sales WHERE year = ms.year AND month = ms.month), 0) * 100 AS sales_percentage,
    (SELECT AVG(avg_order_value) FROM monthly_sales WHERE year = ms.year AND month = ms.month) AS overall_avg_order_value
FROM monthly_sales ms

UNION ALL

SELECT 
    'Category Performance' AS report_type,
    cs.year,
    cs.month,
    NULL AS region,
    cs.category_name AS customer_segment,
    cs.unique_buyers AS unique_customers,
    cs.order_count AS total_orders,
    cs.total_sales AS gross_sales,
    cs.total_sales / NULLIF(cs.order_count, 0) AS avg_order_value,
    NULL AS cancelled_orders,
    (SELECT SUM(total_sales) FROM sub_category_sales WHERE year = cs.year AND month = cs.month) AS total_monthly_sales,
    cs.total_sales / NULLIF((SELECT SUM(total_sales) FROM category_sales WHERE year = cs.year AND month = cs.month), 0) * 100 AS sales_percentage,
    NULL AS overall_avg_order_value
FROM category_sales cs
LIMIT 50, 100"""

    sql_len = len(sql)
    print("sql length: ", sql_len)

    # parse sql statements to SQL object
    sql_stmt = Parsed(sql)
    # Format and print the SQL statement with proper indentation
    print(sql_stmt.format())  # Output formatted SQL statement

    # Tokenization - returns list of tuples containing (token_value, token_type, position)
    tokens = ParsedQuery.tokenize(sql)  # Get tuple list of token information (token_value, token_type, position)

    # Alternative tokenization - returns list of token objects with attributes
    token_obj_list = sql_stmt.tokens()  # Get object list of token information

    # Generate and print Abstract Syntax Tree (AST) in JSON format
    print(sql_stmt.AST())  # Get JSON structure of the SQL statement

    # Extract table lineage/dependencies from the query
    src_tables = ParsedQuery.parse_dependence(sql)  # Get source tables (dependencies) of the query

Comment Handling (pure)

pure controls SQL comment handling in parser constructors such as Parsed, ParsedQuery, ParsedInsert, ParsedCTE, ParsedUpdate, ParsedDelete, ParsedView, and ParsedCreate.

  • pure=False (default): keep comments in parsing/formatting output.
  • pure=True: strip -- and /* ... */ comments before parsing; formatted output and token results exclude comments, and parsing may be faster.
from fastsqlparse import Parsed

parsed_keep = Parsed(sql, pure=False)  # preserve comments
parsed_clean = Parsed(sql, pure=True)  # strip comments before parse

Dialects (dialect)

dialect selects the SQL dialect for parsing and lexical analysis (default "ansi"). Every parser constructor accepts dialect as a string; tokenize classmethods and ParsedQuery.parse_dependence accept a DialectType (default DialectType.ANSI).

Supported dialects: ansi, mysql, postgresql, sqlite, doris (see the Dialects enum and the DIALECT_* constants in fastsqlparse.conf).

from fastsqlparse import Parsed, ParsedQuery, Dialects, DialectType

parsed = Parsed(sql, dialect=Dialects.MYSQL.value)          # "mysql"
query = ParsedQuery(sql, "q", dialect="postgresql")
ParsedQuery.tokenize(sql, dialect=DialectType.MYSQL)        # typed DialectType
ParsedQuery.parse_dependence(sql, dialect="mysql")

When to Use Which Parser

Scenario Parser to Use
SQL statement type is unknown or you don't want to specify the type Parsed/ParsedOne
Multiple SQL statements separated by ; (script execution) Parsed
SELECT / query statement ParsedQuery
INSERT statement ParsedInsert
DELETE statement ParsedDelete
UPDATE statement ParsedUpdate
CREATE TABLE statement ParsedCreate
CREATE VIEW statement ParsedView
CTE (WITH clause) statement ParsedCTE

Note: If your SQL contains multiple statements separated by semicolons (e.g., a script with CREATE, INSERT, SELECT), you must use Parsed. The type-specific parsers are designed for single, known-type statements only.

Documentation

For complete API documentation, see: API_DOC.md

Contributing

Contributions are welcome! Please feel free to submit pull requests, report bugs, or suggest new features.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Note on Dynamic Libraries

This project currently distributes precompiled dynamic libraries (.pyd and .so). The corresponding C++ source code for these dynamic libraries is temporarily not public and is planned to be opened in a future release.

For the full supplementary notice, see LICENSE.

You can also use the dynamic libraries from the source code directly to develop your own SQL parsing library.

Project links

Data verified by PyPI on Aug 29, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for Nohaltsail from gravatar.com Nohaltsail

Credits

Author: NohaltSial

GitHub Statistics

Data verified by PyPI on Aug 29, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.10

Tags

sql parser ast fast

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

fast_pysqlparse-0.8.3-cp314-cp314-win_amd64.whl (1.6 MB view details)

Uploaded CPython 3.14Windows x86-64

fast_pysqlparse-0.8.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.3-cp313-cp313-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.13Windows x86-64

fast_pysqlparse-0.8.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.3-cp312-cp312-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.12Windows x86-64

fast_pysqlparse-0.8.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.3-cp311-cp311-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.11Windows x86-64

fast_pysqlparse-0.8.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.3-cp310-cp310-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.10Windows x86-64

fast_pysqlparse-0.8.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

File details

Details for the file fast_pysqlparse-0.8.3-cp314-cp314-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 7fb0af2af03880424eb12e951d827aa3b5c536660eec3056750a11584b6a3371
MD5 fd91a097a91cc48e9c5af17157772955
BLAKE2b-256 5dfd98909dc665f64b251b1c78df2d0bc20b2c80758f8136263bebc30aa0e0d0

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp314-cp314-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 e06f616d791e31bbafbe1bd89291e72fe930fecd0d1fbae5c295c4c5da8ab8e4
MD5 9d039fa344c8c369f622571768f37cf8
BLAKE2b-256 081eb545b672a27390c1cb04faf3d82d0cc983f490780cd08b83904c6966a741

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp313-cp313-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 a71ab9522e7078ded5b53e249788d2a273f180bc98c2827f1784c9a0de9d439c
MD5 c6ecff045a0960713a701d11a298dd6a
BLAKE2b-256 17a87d0b174885f3844df2db7613346195df53757f1a6771e41dfc2b81a2d77d

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp313-cp313-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 05affffd5ff2246fc0dbc6f276e25b16f91814d4df09ac588929384d495ab75b
MD5 e4c03727a68f49e79c1a3e114b5478b1
BLAKE2b-256 421b4a2991e9269c96eae96f6905563f76c47cf2fd94e6deaa3d940ebe928c8f

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 b82014fb6029c89cbef8c2631f5ee9fdde8ee6f92b773a12d1748dd3685d3c76
MD5 8d3e9a2632aa3ba86fd3e2486af7e7dd
BLAKE2b-256 d9dcf09e2e8edd91f3b786976375618d8c96b312dc8b4a9da5b06de0c33ab65b

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp312-cp312-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 ef334fa8bc5e43cf633c32989d82837e72cb5cd07cba116cbd3d4c769142490e
MD5 511898f857899d0e35adcea34d1c86c5
BLAKE2b-256 f5b815f1d5a4f1050c0a39c00e2b0a5e134047cb43b46e828f932835058b8edf

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 7be6d8d0595847b5a9eea9a7b04318212386f12ebaad33e5ad12d0ac211011d5
MD5 bbd5ea1caec4bd211f1ad18b7cc934ee
BLAKE2b-256 f02c75ecb553b864de9b96179a404bb0c4df446f2e18cb35e7a0c9924aee2e10

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp311-cp311-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 0c736d7516e8906fb49a9daa0971da7293a3cb97189612318f30370d75efd0a9
MD5 3fe43c7efc700356f055617c7cdde79b
BLAKE2b-256 cfa897c2645eadb53daedfc359126cc7c874b181411d22a3fc452aa7870d8776

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 de82f71d8e3d66e49b7e340f1354b31071afc9de2a1315426bddb63872935b2e
MD5 73b8bb08af0c293d6576cdb7a6eb0aa1
BLAKE2b-256 ef83acc94c78bfe46854b5478b7ed9d20bed80b993a4cdc1a9c752f315358ebe

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp310-cp310-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 364524f7f93baad5392752062666f0ed3dbc8df1689669802ac60c8a9fe62606
MD5 8377618905c9e58011fbe605566952d8
BLAKE2b-256 769f2b7128a514c3b65dae977e6373bc324e2fc28cdc206c967c25a1a1de125e

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.8.3 This release

10 files

0.8.2

10 files

0.8.1

10 files

0.8.0

10 files

0.7.1

10 files

0.7.0

10 files

0.6.1

10 files

0.6.0

2 files

0.5.1

2 files

0.5.0

2 files

0.4.0

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.9

2 files

0.2.8

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.9

2 files

0.1.8

2 files

0.1.7

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page