SQL Lineage Analysis Tool powered by Python
Project description
SQLLineage
SQL Lineage Analysis Tool powered by Python
Never get the hang of a SQL parser? SQLLineage comes to the rescue. Given a SQL command, SQLLineage will tell you its source and target tables, without worrying about Tokens, Keyword, Identifier and all the jagons used by SQL parsers.
Behind the scene, SQLLineage uses the fantastic sqlparse
library to parse
the SQL command, and bring you all the human-readable result with ease.
Quick Start
Install sqllineage via PyPI:
$ pip install sqllineage
Using sqllineage command to parse a quoted-query-string:
$ sqllineage -e "insert into db1.table1 select * from db2.table2"
Statements(#): 1
Source Tables:
db2.table2
Target Tables:
db1.table1
Or you can parse a SQL file with -f option:
$ sqllineage -f foo.sql
Statements(#): 1
Source Tables:
db1.table_foo
db1.table_bar
Target Tables:
db2.table_baz
Advanced Usage
Lineage result combined for multiple SQL statements, with intermediate tables identified:
$ sqllineage -e "insert into db1.table1 select * from db2.table2; insert into db3.table3 select * from db1.table1;"
Statements(#): 2
Source Tables:
db2.table2
Target Tables:
db3.table3
Intermediate Tables:
db1.table1
And if you want to see lineage result for every SQL statement, just toggle verbose option
$ sqllineage -v -e "insert into db1.table1 select * from db2.table2; insert into db3.table3 select * from db1.table1;"
Statement #1: insert into db1.table1 select * from db2.table2;
table read: {Table: db2.table2}
table write: {Table: db1.table1}
table rename: {}
table drop: {}
table intermediate: {}
Statement #2: insert into db3.table3 select * from db1.table1;
table read: {Table: db1.table1}
table write: {Table: db3.table3}
table rename: {}
table drop: {}
table intermediate: {}
==========
Summary:
Statements(#): 2
Source Tables:
db2.table2
Target Tables:
db3.table3
Intermediate Tables:
db1.table1
Or specify your own combiner to combine lineage result, here we use a naive combiner, which is just to union each table set without removing intermediate tables:
$ sqllineage -e "insert into db1.table1 select * from db2.table2; insert into db3.table3 select * from db1.table1;" -c sqllineage.combiners.NaiveLineageCombiner
Statements(#): 2
Source Tables:
db2.table2
db1.table1
Target Tables:
db3.table3
db1.table1
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for sqllineage-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8fda5027424a5057ecad76a3eab0f9369ac9f3c24a8e9c0f03f8977e06bfe5db |
|
MD5 | 937e3d01bfa6501ec060b8f9c402aba2 |
|
BLAKE2b-256 | 807ca7a32cb03475dba596af531f5dec80234487773237dffed8fca20cb26a67 |