Skip to main content

PostgreSQL Languages AST and statements prettifier

Project description

Author: Lele Gaifax
Contact: lele@metapensiero.it
License:GNU General Public License version 3 or later
Status: Build status Documentation status Test coverage status
Version: 3

This is a Python 3 module that exposes the parse tree of a PostgreSQL statement (extracted by the almost standard PG parser repackaged as a standalone static library by libpg_query) as set of interconnected nodes, usually called an abstract syntax tree.

See a more detailed introduction in the documentation.

Installation

As usual, the easiest way is with pip:

$ pip install pglast

Alternatively you can clone the repository:

$ git clone https://github.com/lelit/pglast.git --recursive

and install from there:

$ pip install ./pglast

Development

There is a set of makefiles implementing the most common operations, a make help will show a brief table of contents. A comprehensive test suite, based on pytest, covers nearly 99% of the source lines.

Documentation

Latest documentation is hosted by Read the Docs at http://pglast.readthedocs.io/

Changes

Version 3

3.12 (2022-06-19)

  • Rewrite the implementation of the referenced_relations() function, that was flawed with regard to CTEs handling (issue 106), thanks to Michal Charemza for providing his own version
  • Improve WithClause printer indentation
  • Fix minor whitespace related issues in a few printer functions

3.11 (2022-05-29)

  • Fix the Visitor class, it was ignoring nodes nested in sub-lists
  • Reduce the size of the generated parser by factoring out common code into helper functions

3.10 (2022-05-11)

3.9 (2022-02-24)

  • Fix bug handling node containing a location field, e.g. CreateTableSpaceStmt (issue #98)
  • Properly handle dereferenced array expression (issue #99)
  • Avoid improper “floatification” of literal integers (issue #100)

3.8 (2021-12-28)

  • Fix glitch in the AST extractor tool (issue #97)
  • Add Linux AArch64 wheel build support (PR #95), thanks to odidev
  • Fix type mismatch when using --remove-pg_catalog-from-functions (PR #93), thanks to Boris Zentner

3.7 (2021-10-13)

3.6 (2021-10-09)

  • Use latest libpg_query, to fix an error parsing PLpgSQL statements (issue #88)

3.5 (2021-09-26)

  • Forward the special_functions option to substream, when concatenating items (issue #89)
  • Fix representation of floating point numbers without decimal digits (issue #91)
  • Produce Python 3.10 wheels, thanks to cibuildwheel 2.1.2
  • Update libpg_query to 13-2.0.7
  • New option --remove-pg_catalog-from-functions on the command line tool (PR #90), thanks to Boris Zentner
  • Implement more special functions (PR #92), thanks to Boris Zentner

3.4 (2021-08-21)

  • Fix another packaging issue, that prevented recompilation from the sdist .tar.gz (issue #86), thanks to Christopher Brichford

3.3 (2021-07-04)

3.2 (2021-06-25)

  • Effectively include libpg_query’s vendored sources (issue #82)

3.1 (2021-06-25)

  • Fix packaging glitch (issue #82)
  • Build wheels also for macOS
  • Update libpg_query to 13-2.0.5

3.0 (2021-06-04)

  • Fix glitch in the RawStream, avoiding spurious space after an open parenthesis
  • Improve the Visitor class, to make it easier altering the original tree
  • Properly handle nested lists in the serialization of AST Node

3.0.dev2 (2021-05-22)

  • Fix bug in CreateStmt printer (issue #79)
  • Make it possible to pass also concrete ast.Nodes to RawStream`
Breaking changes
  • To reduce confusion, the printer module has been removed: print-specific stuff is now directly exposed by the printers subpackage while serialization classes are now in the new stream module
  • The default value for the safety_belt option of the printify() function is now False

3.0.dev1 (2021-05-16)

  • Fix AT_SetIdentity, AT_EnableReplicaTrig and AlterSubscriptionStmt printers
  • Improve AlterTSConfigType and IntoClause printers
  • New generic “visitor pattern” (issue #51) exemplified by a new referenced_relations() function (issue #66)
  • Refine printing of SQL comments
  • Implement AlterExtensionStmt printer

3.0.dev0 (2021-05-03)

  • Expose the new pg_query_scan() function as parser.scan()
  • Expose the pg_query_parse() function as parser.parse_sql_json()
  • Expose the new pg_query_parse_protobuf() function as parser.parse_sql_protobuf()
  • Expose the new pg_query_deparse_protobuf() function as parser.deparse_protobuf()
  • Honor the catalogname of a RangeVar if present (issue #71)
  • Cover almost all SQL statements, testing against the whole PostgreSQL regression suite (issue #68, PR #72 and PR #77), thanks to Ronan Dunklau and Hong Cheng
  • New rudimentary support for the preserve comments feature (issue #23)
Breaking changes
  • Target PostgreSQL 13
  • The pglast.parser module exposes all libpg_query entry points, even the new pg_query_deparse_protobuf() function that is basically equivalent to RawStream-based printer
  • The split() function is now based on the lower level pg_query_split_with_xxx() functions
  • The parse_sql() function returns native Python objects, not a JSON string as before: all PG nodes are now represented by subclasses of pglast.ast.Node, without exception, even Expr and Value are there. The latter impacts on pglast.node.Scalar: for example it now may contains a ast.Integer instance instead of a Python int
  • The pgpp --parse-tree output is a pprint represention of the AST, not a JSON string as before
  • The ParseError exception does not expose the location as an instance member anymore, although its still there, as the second argument (ie .args[1]); furthermore, its value now corresponds to the index in the original Unicode string, instead of the offset in the UTF-8 representation passed to the underlying C function

Version 2

2.0.dev3 (2021-02-20)

  • Handle INCLUDE clause in IndexStmt (PR #67), thanks to Ronan Dunklau

2.0.dev2 (2020-10-24)

  • Merge new fingerprint functionality from v1 (i.e. master) branch

2.0.dev1 (2020-09-26)

  • Require Python 3.6 or greater
  • Handle ALTER TYPE .. RENAME VALUE in AlterEnumStmt (PR #52), thanks to Ronan Dunklau
  • Add support for Create / Alter / Drop PROCEDURE (PR #48), thanks to Ronan Dunklau
  • Use Ronan’s fork of libpg_query, targeting PostgreSQL 12.1 (PR #36)
  • Change get_postgresql_version() to return a (major, minor) tuple (issue #38)
  • Handle ALTER TABLE ... ALTER COLUMN ... SET STORAGE ...
  • Handle PG12 materialized CTEs (issue #57)
  • Support column numbers in ALTER INDEX (PR #58), thanks to Ronan Dunklau
  • Handle SET LOGGED and SET UNLOGGED in ALTER TABLE (PR #59), thanks to Ronan Dunklau
  • Handle ALTER TYPE ... RENAME (PR #62), , thanks to Ronan Dunklau

Version 1

1.18 (2021-06-01)

  • Fix exclusion constraint printer (issue #81)

1.17 (2021-02-20)

  • Fix the generic case in the RenameStmt printer

1.16 (2021-02-20)

  • Promote to the stable state
  • Move the job of building and uploading binary wheels from TravisCI to GitHub Actions

1.15 (2021-02-19)

  • Fix IF EXISTS variant of RenameStmt printer (PR #70), thanks to Jonathan Mortensen
  • Update libpg_query to 10-1.0.5

1.14 (2020-10-24)

  • Produce Python 3.9 wheels, thanks to cibuildwheel 1.6.3
  • Expose the libpg_query’s fingerprint functionality (PR #64), thanks to Yiming Wang

1.13 (2020-09-26)

  • Handle SELECT FROM foo

1.12 (2020-06-08)

  • Double quote column names in the TYPE_FUNC_NAME_KEYWORDS set (issue #55)
  • Possibly wrap SELECT in UNION/INTERSECT between parens, when needed (issue #55)

1.11 (2020-05-08)

  • Fix A_Expr printer, when lexpr is missing (PR #54), thanks to Aiham
  • Support DISABLE ROW LEVEL SECURITY in AlterTableCmd (PR #49), thanks to Ronan Dunklau
  • Implement CreateOpClassStmt printer (PR #47), thanks to Ronan Dunklau

1.10 (2020-01-25)

  • Fix collation name printer (PR #44), thanks to Ronan Dunklau
  • Implement CreatePLangStmt printer (PR #42), thanks to Bennie Swart
  • Fix privileges printer (PR #41), thanks to Bennie Swart
  • Handle TRUNCATE event in CreateTrigStmt printer (PR #40), thanks to Bennie Swart
  • Fix function body dollar quoting (PR #39), thanks to Bennie Swart

1.9 (2019-12-20)

  • Prettier INSERT representation

1.8 (2019-12-07)

  • Prettier CASE representation
  • New option to emit a semicolon after the last statement (issue #24)

1.7 (2019-12-01)

  • Implement NotifyStmt printer
  • Implement RuleStmt printer, thanks to Gavin M. Roy for his PR #28
  • Fix RenameStmt, properly handling object name
  • Produce Python 3.8 wheels, thanks to cibuildwheel 1.0.0
  • Support ALTER TABLE RENAME CONSTRAINT (PR #35), thanks to Ronan Dunklau

1.6 (2019-09-04)

  • Fix issue with boolean expressions precedence (issue #29)
  • Implement BitString printer
  • Support LEAKPROOF option (PR #31), thanks to Ronan Dunklau
  • Support DEFERRABLE INITIALLY DEFERRED option (PR #32), thanks to Ronan Dunklau

1.5 (2019-06-04)

  • Fix issue with RETURNS SETOF functions, a more general solution than the one proposed by Ronan Dunklau (PR #22)
  • Allow more than one empty line between statements (PR #26), thanks to apnewberry

1.4 (2019-04-06)

  • Fix wrap of trigger’s WHEN expression (issue #18)
  • Support for variadic functions (PR #19), thanks to Ronan Dunklau
  • Support ORDER / LIMIT / OFFSET for set operations (PR #20), thanks to Ronan Dunklau
  • Implement ConstraintsSetStmt and improve VariableSetStmt printers

1.3 (2019-03-28)

  • Support CROSS JOIN and timezone modifiers on time and timestamp datatypes (PR #15), thanks to Ronan Dunklau
  • Many new printers and several enhancements (PR #14), thanks to Ronan Dunklau
  • Expose the package version as pglast.__version__ (issue #12)

1.2 (2019-02-13)

  • Implement new split() function (see PR #10)
  • Implement BooleanTest printer (issue #11)

1.1 (2018-07-20)

  • No visible changes, but now PyPI carries binary wheels for Python 3.7.

1.0 (2018-06-16)

Important

The name of the package has been changed from pg_query to pglast, to satisfy the request made by the author of libpg_query in issue #9.

This affects both the main repository on GitHub, that from now on is https://github.com/lelit/pglast, and the ReadTheDocs project that hosts the documentation, http://pglast.readthedocs.io/en/latest/.

I’m sorry for any inconvenience this may cause.

0.28 (2018-06-06)

  • Update libpg_query to 10-1.0.2
  • Support the ‘?’-style parameter placeholder variant allowed by libpg_query (details)

0.27 (2018-04-15)

  • Prettier JOINs representation, aligning them with the starting relation

0.26 (2018-04-03)

  • Fix cosmetic issue with ANY() and ALL()

0.25 (2018-03-31)

  • Fix issue in the safety belt check performed by pgpp (issue #4)

0.24 (2018-03-02)

  • Implement Null printer

0.23 (2017-12-28)

  • Implement some other DDL statements printers
  • New alternative style to print comma-separated-values lists, activated by a new --comma-at-eoln option on pgpp

0.22 (2017-12-03)

  • Implement TransactionStmt and almost all DROP xxx printers

0.21 (2017-11-22)

  • Implement NamedArgExpr printer
  • New alternative printers for a set of special functions, activated by a new --special-functions option on pgpp (issue #2)

0.20 (2017-11-21)

  • Handle special de-reference (A_Indirection) cases

0.19 (2017-11-16)

  • Fix serialization of column labels containing double quotes
  • Fix corner issues surfaced implementing some more DDL statement printers

0.18 (2017-11-14)

  • Fix endless loop due to sloppy conversion of command line option
  • Install the command line tool as pgpp

0.17 (2017-11-12)

  • Rename printers.sql to printers.dml (backward incompatibility)
  • List printer functions in the documentation, referencing the definition of related node type
  • Fix inconsistent spacing in JOIN condition inside a nested expression
  • Fix representation of unbound arrays
  • Fix representation of interval data type
  • Initial support for DDL statements
  • Fix representation of string literals containing single quotes

0.16 (2017-10-31)

  • Update libpg_query to 10-1.0.0

0.15 (2017-10-12)

  • Fix indentation of boolean expressions in SELECT’s targets (issue #3)

0.14 (2017-10-09)

  • Update to latest libpg_query’s 10-latest branch, targeting PostgreSQL 10.0 final

0.13 (2017-09-17)

  • Fix representation of subselects requiring surrounding parens

0.12 (2017-08-22)

  • New option --version on the command line tool
  • Better enums documentation
  • Release the GIL while calling libpg_query functions

0.11 (2017-08-11)

  • Nicer indentation for JOINs, making OUTER JOINs stand out
  • Minor tweaks to lists rendering, with less spurious whitespaces
  • New option --no-location on the command line tool

0.10 (2017-08-11)

  • Support Python 3.4 and Python 3.5 as well as Python 3.6

0.9 (2017-08-10)

  • Fix spacing before the $ character
  • Handle type modifiers
  • New option --plpgsql on the command line tool, just for fun

0.8 (2017-08-10)

  • Add enums subpackages to the documentation with references to their related headers
  • New compact_lists_margin option to produce a more compact representation when possible (see issue #1)

0.7 (2017-08-10)

  • Fix sdist including the Sphinx documentation

0.6 (2017-08-10)

  • New option --parse-tree on the command line tool to show just the parse tree
  • Sphinx documentation, available online

0.5 (2017-08-09)

  • Handle some more cases when a name must be double-quoted
  • Complete the serialization of the WindowDef node, handling its frame options

0.4 (2017-08-09)

  • Expose the actual PostgreSQL version the underlying libpg_query libray is built on thru a new get_postgresql_version() function
  • New option safety_belt for the prettify() function, to protect the innocents
  • Handle serialization of CoalesceExpr and MinMaxExpr

0.3 (2017-08-07)

  • Handle serialization of ParamRef nodes
  • Expose a prettify() helper function

0.2 (2017-08-07)

  • Test coverage at 99%
  • First attempt at automatic wheel upload to PyPI, let’s see…

0.1 (2017-08-07)

  • First release (“Hi daddy!”, as my soul would tag it)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pglast-3.12.tar.gz (2.9 MB view hashes)

Uploaded source

Built Distributions

pglast-3.12-cp37-cp37m-macosx_10_9_x86_64.whl (989.7 kB view hashes)

Uploaded cp37

pglast-3.12-cp36-cp36m-macosx_10_9_x86_64.whl (988.0 kB view hashes)

Uploaded cp36

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page