Extract Parse Tree from SQL
Project description
Moz SQL Parser
==============
Let's make a SQL parser so we can provide a familiar interface to
non-sql datastores!
+----------+------------------+
| Branch | Status |
+==========+==================+
| master | |Build Status| |
+----------+------------------+
| dev | |Build Status| |
+----------+------------------+
Problem Statement
-----------------
SQL is a familiar language used to access databases. Although, each
database vendor has its quirky implementation, the average developer
does not know enough SQL to be concerned with those quirks. This
familiar core SQL (lowest common denominator, if you will) is useful
enough to explore data in primitive ways. It is hoped that, once
programmers have reviewed a datastore with basic SQL queries, and they
see the value of that data, they will be motivated to use the
datastore's native query format.
Objectives
----------
The primary objective of this library is to convert some subset of
`SQL-92 <https://en.wikipedia.org/wiki/SQL-92>`__ queries to JSON-izable
parse trees. A big enough subset to provide superficial data access via
SQL, but not so much as we must deal with the document-relational
impedance mismatch.
Non-Objectives
--------------
- No plans to provide update statements, like ``update`` or ``insert``
- No plans to expand the language to all of SQL:2011
- No plans to provide data access tools
Project Status
--------------
There are `over 160 tests, all
passing <https://github.com/mozilla/moz-sql-parser/tree/dev/tests>`__.
This parser is good enough for basic usage, including inner queries.
You can see the parser in action at https://sql.telemetry.mozilla.org/
while using the ActiveData datasource
Install
-------
::
pip install moz-sql-parser
Usage
-----
::
>>> from moz_sql_parser import parse
>>> import json
>>> json.dumps(parse("select count(1) from jobs"))
'{"from": "jobs", "select": {"value": {"count": {"literal": 1}}}}'
Each SQL query is parsed to an object: Each clause is assigned to an
object property of the same name.
::
>>> json.dumps(parse("select a as hello, b as world from jobs"))
'{"from": "jobs", "select": [{"name": "hello", "value": "a"}, {"name": "world", "value": "b"}]}'
The ``SELECT`` clause is an array of objects containing ``name`` and
``value`` properties.
Run Tests
---------
See `the tests
directory <https://github.com/mozilla/moz-sql-parser/tree/dev/tests>`__
for instructions running tests, or writing new ones.
More about implementation
-------------------------
SQL queries are translated to JSON objects: Each clause is assigned to
an object property of the same name.
::
# SELECT * FROM dual WHERE a>b ORDER BY a+b
{
"select": "*",
"from": "dual"
"where": {"gt": ["a","b"]},
"orderby": {"add": ["a", "b"]}
}
Expressions are also objects, but with only one property: The name of
the operation, and the value holding (an array of) parameters for that
operation.
::
{op: parameters}
and you can see this pattern in the previous example:
::
{"gt": ["a","b"]}
Notes
~~~~~
- Uses the glorious ``pyparsing`` library (see
http://pyparsing.wikispaces.com/) to define the grammar, and define
the shape of the tokens it generates.
- ``[sqlparse](https://pypi.python.org/pypi/sqlparse)`` Does not
provide a tree, rather a list of tokens.
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=master
:target: https://travis-ci.org/mozilla/moz-sql-parser
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=dev
:target: https://travis-ci.org/mozilla/moz-sql-parser
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=master
:target: https://travis-ci.org/mozilla/moz-sql-parser
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=dev
:target: https://travis-ci.org/mozilla/moz-sql-parser
==============
Let's make a SQL parser so we can provide a familiar interface to
non-sql datastores!
+----------+------------------+
| Branch | Status |
+==========+==================+
| master | |Build Status| |
+----------+------------------+
| dev | |Build Status| |
+----------+------------------+
Problem Statement
-----------------
SQL is a familiar language used to access databases. Although, each
database vendor has its quirky implementation, the average developer
does not know enough SQL to be concerned with those quirks. This
familiar core SQL (lowest common denominator, if you will) is useful
enough to explore data in primitive ways. It is hoped that, once
programmers have reviewed a datastore with basic SQL queries, and they
see the value of that data, they will be motivated to use the
datastore's native query format.
Objectives
----------
The primary objective of this library is to convert some subset of
`SQL-92 <https://en.wikipedia.org/wiki/SQL-92>`__ queries to JSON-izable
parse trees. A big enough subset to provide superficial data access via
SQL, but not so much as we must deal with the document-relational
impedance mismatch.
Non-Objectives
--------------
- No plans to provide update statements, like ``update`` or ``insert``
- No plans to expand the language to all of SQL:2011
- No plans to provide data access tools
Project Status
--------------
There are `over 160 tests, all
passing <https://github.com/mozilla/moz-sql-parser/tree/dev/tests>`__.
This parser is good enough for basic usage, including inner queries.
You can see the parser in action at https://sql.telemetry.mozilla.org/
while using the ActiveData datasource
Install
-------
::
pip install moz-sql-parser
Usage
-----
::
>>> from moz_sql_parser import parse
>>> import json
>>> json.dumps(parse("select count(1) from jobs"))
'{"from": "jobs", "select": {"value": {"count": {"literal": 1}}}}'
Each SQL query is parsed to an object: Each clause is assigned to an
object property of the same name.
::
>>> json.dumps(parse("select a as hello, b as world from jobs"))
'{"from": "jobs", "select": [{"name": "hello", "value": "a"}, {"name": "world", "value": "b"}]}'
The ``SELECT`` clause is an array of objects containing ``name`` and
``value`` properties.
Run Tests
---------
See `the tests
directory <https://github.com/mozilla/moz-sql-parser/tree/dev/tests>`__
for instructions running tests, or writing new ones.
More about implementation
-------------------------
SQL queries are translated to JSON objects: Each clause is assigned to
an object property of the same name.
::
# SELECT * FROM dual WHERE a>b ORDER BY a+b
{
"select": "*",
"from": "dual"
"where": {"gt": ["a","b"]},
"orderby": {"add": ["a", "b"]}
}
Expressions are also objects, but with only one property: The name of
the operation, and the value holding (an array of) parameters for that
operation.
::
{op: parameters}
and you can see this pattern in the previous example:
::
{"gt": ["a","b"]}
Notes
~~~~~
- Uses the glorious ``pyparsing`` library (see
http://pyparsing.wikispaces.com/) to define the grammar, and define
the shape of the tokens it generates.
- ``[sqlparse](https://pypi.python.org/pypi/sqlparse)`` Does not
provide a tree, rather a list of tokens.
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=master
:target: https://travis-ci.org/mozilla/moz-sql-parser
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=dev
:target: https://travis-ci.org/mozilla/moz-sql-parser
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=master
:target: https://travis-ci.org/mozilla/moz-sql-parser
.. |Build Status| image:: https://travis-ci.org/mozilla/moz-sql-parser.svg?branch=dev
:target: https://travis-ci.org/mozilla/moz-sql-parser
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
moz-sql-parser-2.17.18212.tar.gz
(14.0 kB
view hashes)
Close
Hashes for moz-sql-parser-2.17.18212.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2cdb87994034e7acb76881139b8ab0e0236121ea8aef62ac4decf5e0be40c018 |
|
MD5 | fbdb01e1f1f7162334131bb5198fb82f |
|
BLAKE2b-256 | 6865de981c4e774316b25f36bcb20e1e501096f0f16d338e9d67bcced5385f9c |