fast-pysqlparse: High-Performance SQL Parsing Library

Build Status Version Language License

README.md (Chinese)

A high-performance, cross-platform, lightweight SQL parsing library whose core trait is speed — built on a C++17 core with native Python bindings, it rapidly performs structured parsing of SQL, especially statements with highly complex structure and deep nesting.

Overview

fast-pysqlparse aims to overcome the performance and capability limits of traditional Python SQL parsers. By moving compute-intensive parsing into a native C++ layer, it maintains fast parsing even on large, deeply nested, structurally complex SQL.

The parser primarily parses ANSI-style SQL. For statements with a specific dialect, specifying the dialect is recommended; supported dialects: MySQL, PostgreSQL, SQLite, Doris.

Features

  • Fast SQL Parsing: Leverages a high-performance C++17 core to parse SQL statements rapidly
  • Structured Parsing: Oriented toward complex structure and deep nesting, especially statements mixing CTEs with SELECT/INSERT/VIEW
  • Cross-Platform: Compiled into native extensions (.pyd for Windows, .so for Linux)
  • Comprehensive SQL Support: Supports a wide range of SQL statements, including:
    • SELECT (with complex JOIN, WHERE, GROUP BY, subqueries, etc.)
    • INSERT
    • Data Definition Language (CREATE)
    • VIEW
    • DELETE
    • UPDATE
    • Common Table Expressions (CTEs), including nested CTEs
  • Abstract Syntax Tree (AST): Generates a detailed JSON representation of the parsed SQL AST for easy traversal and analysis
  • SQL Formatting: Automatically reformats messy SQL into a clean, readable structure
  • Table Lineage Parsing: Automatically traces and reveals the source-to-target relationships between tables (data lineage)
  • Tokenization: Breaks down SQL statements into their fundamental tokens for lexical analysis
  • Python API: A clean and intuitive Python library built around the high-speed native extension

Performance

This library is engineered for speed. By moving the computationally intensive parsing work to a native C++ layer, it significantly outperforms pure-Python parsing libraries, especially when dealing with large, complex SQL scripts.

Benchmark Results

Test 1: 5000 Iterations

  • SQL Length: 639 characters
  • Total Time: 0.48s
  • PPS (Parses Per Second): ~10300
  • Average per parse: ~0.097ms

Test 2: 10 Million Character SQL

  • SQL Length: 10,500,998 characters
  • Total Time: 0.54s
  • CPS (Characters Per Second): ~19,500,000
  • Parse successful!

Test 3: Python-Only Comparison on ~10M PostgreSQL SQL (No C Benchmark)

Benchmark script: test/python_parsers_10m.py

Parser Avg Time CPS
fastsqlparse 0.7394s 13,524,892.44
pglast 4.8541s 2,060,217.57
sqlglot (postgres) 20.5364s 486,963.12
sqlparse 84.9591s 117,709.35

Notes:

  • SQL size: 10,000,484 chars
  • Runs per parser: 1
  • Results source: results of test/python_parsers_10m.py

Installation

pip install fast-pysqlparse

From Source:

git clone https://github.com/Nohaltsail/fast-pysqlparse.git
cd fast-pysqlparse
pip install build
python -m build
cd dist
pip install fast_pysqlparse-*.whl

Quick Start

from fastsqlparse import Parsed, ParsedQuery

if __name__ == '__main__':
    sql = """

-- main query
SELECT 
    'Monthly Sales Report' AS report_type,
    ms.year,
    ms.month,
    ms.region,
    ms.customer_segment,
    ms.unique_customers,
    ms.total_orders,
    ms.gross_sales,
    ms.avg_order_value,
    ms.cancelled_orders,
    (SELECT SUM(gross_sales) FROM sub_monthly_sales WHERE year = ms.year AND month = ms.month) AS total_monthly_sales,
    ms.gross_sales / NULLIF((SELECT SUM(gross_sales) FROM monthly_sales WHERE year = ms.year AND month = ms.month), 0) * 100 AS sales_percentage,
    (SELECT AVG(avg_order_value) FROM monthly_sales WHERE year = ms.year AND month = ms.month) AS overall_avg_order_value
FROM monthly_sales ms

UNION ALL

SELECT 
    'Category Performance' AS report_type,
    cs.year,
    cs.month,
    NULL AS region,
    cs.category_name AS customer_segment,
    cs.unique_buyers AS unique_customers,
    cs.order_count AS total_orders,
    cs.total_sales AS gross_sales,
    cs.total_sales / NULLIF(cs.order_count, 0) AS avg_order_value,
    NULL AS cancelled_orders,
    (SELECT SUM(total_sales) FROM sub_category_sales WHERE year = cs.year AND month = cs.month) AS total_monthly_sales,
    cs.total_sales / NULLIF((SELECT SUM(total_sales) FROM category_sales WHERE year = cs.year AND month = cs.month), 0) * 100 AS sales_percentage,
    NULL AS overall_avg_order_value
FROM category_sales cs
LIMIT 50, 100"""

    sql_len = len(sql)
    print("sql length: ", sql_len)

    # parse sql statements to SQL object
    sql_stmt = Parsed(sql)
    # Format and print the SQL statement with proper indentation
    print(sql_stmt.format())  # Output formatted SQL statement

    # Tokenization - returns list of tuples containing (token_value, token_type, position)
    tokens = ParsedQuery.tokenize(sql)  # Get tuple list of token information (token_value, token_type, position)

    # Alternative tokenization - returns list of token objects with attributes
    token_obj_list = sql_stmt.tokens()  # Get object list of token information

    # Generate and print Abstract Syntax Tree (AST) in JSON format
    print(sql_stmt.AST())  # Get JSON structure of the SQL statement

    # Extract table lineage/dependencies from the query
    src_tables = ParsedQuery.parse_dependence(sql)  # Get source tables (dependencies) of the query

Comment Handling (pure)

pure controls SQL comment handling in parser constructors such as Parsed, ParsedQuery, ParsedInsert, ParsedCTE, ParsedUpdate, ParsedDelete, ParsedView, and ParsedCreate.

  • pure=False (default): keep comments in parsing/formatting output.
  • pure=True: strip -- and /* ... */ comments before parsing; formatted output and token results exclude comments, and parsing may be faster.
from fastsqlparse import Parsed

parsed_keep = Parsed(sql, pure=False)  # preserve comments
parsed_clean = Parsed(sql, pure=True)  # strip comments before parse

Dialects (dialect)

dialect selects the SQL dialect for parsing and lexical analysis (default "ansi"). Every parser constructor accepts dialect as a string; tokenize classmethods and ParsedQuery.parse_dependence accept a DialectType (default DialectType.ANSI).

Supported dialects: ansi, mysql, postgresql, sqlite, doris (see the Dialects enum and the DIALECT_* constants in fastsqlparse.conf).

from fastsqlparse import Parsed, ParsedQuery, Dialects, DialectType

parsed = Parsed(sql, dialect=Dialects.MYSQL.value)          # "mysql"
query = ParsedQuery(sql, "q", dialect="postgresql")
ParsedQuery.tokenize(sql, dialect=DialectType.MYSQL)        # typed DialectType
ParsedQuery.parse_dependence(sql, dialect="mysql")

When to Use Which Parser

Scenario Parser to Use
SQL statement type is unknown or you don't want to specify the type Parsed/ParsedOne
Multiple SQL statements separated by ; (script execution) Parsed
SELECT / query statement ParsedQuery
INSERT statement ParsedInsert
DELETE statement ParsedDelete
UPDATE statement ParsedUpdate
CREATE TABLE statement ParsedCreate
CREATE VIEW statement ParsedView
CTE (WITH clause) statement ParsedCTE

Note: If your SQL contains multiple statements separated by semicolons (e.g., a script with CREATE, INSERT, SELECT), you must use Parsed. The type-specific parsers are designed for single, known-type statements only.

Documentation

For complete API documentation, see: API_DOC.md

Contributing

Contributions are welcome! Please feel free to submit pull requests, report bugs, or suggest new features.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Note on Dynamic Libraries

This project currently distributes precompiled dynamic libraries (.pyd and .so). The corresponding C++ source code for these dynamic libraries is temporarily not public and is planned to be opened in a future release.

For the full supplementary notice, see LICENSE.

You can also use the dynamic libraries from the source code directly to develop your own SQL parsing library.

Project links

Data verified by PyPI on Aug 28, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.3)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for Nohaltsail from gravatar.com Nohaltsail

Credits

Author: NohaltSial

GitHub Statistics

Data verified by PyPI on Aug 28, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.10

Tags

sql parser ast fast

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

fast_pysqlparse-0.8.2-cp314-cp314-win_amd64.whl (1.6 MB view details)

Uploaded CPython 3.14Windows x86-64

fast_pysqlparse-0.8.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.2-cp313-cp313-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.13Windows x86-64

fast_pysqlparse-0.8.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.2-cp312-cp312-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.12Windows x86-64

fast_pysqlparse-0.8.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.2-cp311-cp311-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.11Windows x86-64

fast_pysqlparse-0.8.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

fast_pysqlparse-0.8.2-cp310-cp310-win_amd64.whl (1.5 MB view details)

Uploaded CPython 3.10Windows x86-64

fast_pysqlparse-0.8.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

File details

Details for the file fast_pysqlparse-0.8.2-cp314-cp314-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 89a995fdd9f445842aae7ce9cb0a46d4e95d66f1bea95e25f5d5e0946b33d878
MD5 31045af358a4b90bd09bca781cdb097d
BLAKE2b-256 29b7ec4f3859120632314bb2ae85c82f2399ebccbdee06e0253c65a11391d91c

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp314-cp314-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 7f39f476f870c6b822f663edb1052d0f040b41f7b651df23cbc0ee67485ac4b9
MD5 30456170e309d24b491fd8a6a0ed9a90
BLAKE2b-256 a67aac8856dee8abfe05e0290a502b1afe4bec9b1075a47caefabaef18cb0057

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp313-cp313-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 8316e3d1df1351f46ce05632ed3c5da34bbd88273024341f5e98ce98b57138be
MD5 b35b6a012e489f398f262cfed57585d9
BLAKE2b-256 d7fa28e9d3422fa6342785e4aa911eaa1460c6d5d4ea144520cc25ee1d581cf0

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp313-cp313-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 503599fd273d11e10e859f53bbb1e8708d3e41cf452ef68811488216fa937c87
MD5 adac75687a9105fda1af6064407480be
BLAKE2b-256 2b389b780705b3f71d7c0e5d206a84fd07222bdc41933d560edcef2cd9ea9057

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 409d4d6f1f505f9b6480fbccd0249384ee53ce64fde86a043b0f9124f0c982a6
MD5 fe90e71931b4ad19c6155c3276fa3ca5
BLAKE2b-256 835557603275e58e1753b23760fa0431381d6f1fbd06f283bc234944fcde2ec8

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp312-cp312-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 7c98cf6f9b25fbc0e8a654f7c3ad5a8ecd1c85dcc41762dfea1db16a85ecd012
MD5 4e0742574832e0305acac7a2ca7de0ea
BLAKE2b-256 3ee58b0ea2fa368294b0ea9550b2293a1c9e9a0eff7743b4ba574127ce359f8d

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 4ac346a3a87a2340b5aaf1e5a2b5e4363f1bab50ca72efbee2bf328c708a27f4
MD5 c46e498dc307ec03413ee2fe1e257a6f
BLAKE2b-256 49006b105d08090c8e4469999e1ee2d9b15da184a57b091f7cb7c84a84a85251

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp311-cp311-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 beea205bc5ab387dbed917c93d321de0e9b56b8608803439920c82989cae5a7e
MD5 2d1a9f09d3869c24c35f44f0e5d01122
BLAKE2b-256 4a0ef8d4deede5aede20020ed092ab173d0bc55f340424847e682d6cf12fe924

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 338eaa76da54af574a79cdc5ce24e077823574d35fcf3d9de2fa843511c1f554
MD5 2c04d18572b7482553e35761d27aeb49
BLAKE2b-256 b99e457b3da96421434706e6c1c299f2b38195c4dfe5c5eee690d54ce3d4f2b8

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp310-cp310-win_amd64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_pysqlparse-0.8.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for fast_pysqlparse-0.8.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 02691b57113cad21e85a497bb487ad1df297686db9363fcc1f66f106d22a5a3f
MD5 a3a3f041de8e4dfdfa43c6dd30e36cb1
BLAKE2b-256 50b78189114e01033ca0bae3b4c9665fe38381bb9359b57d69dda3aea9dcd6f7

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_pysqlparse-0.8.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: python-publish.yml on Nohaltsail/fast-pysqlparse

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.8.3

10 files

This release

0.8.2 This release

10 files

0.8.1

10 files

0.8.0

10 files

0.7.1

10 files

0.7.0

10 files

0.6.1

10 files

0.6.0

2 files

0.5.1

2 files

0.5.0

2 files

0.4.0

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.9

2 files

0.2.8

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.9

2 files

0.1.8

2 files

0.1.7

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page