elementary-lineage 0.0.6

Presenting data lineage based on your data warehouse query history

Data lineage made simple, reliable, and automated.
Effortlessly track the flow of data, understand dependencies and analyze impact.

Features

• Visualization: In browser visual representation of the data lineage graph.
• DWH lineage: Effortlessly map data flow in the data warehouse.
• Accuracy: Reflects the actual state in the DWH.
• Plug-and-play: No need for code changes.

Coming soon:

• Monitoring: Present data about freshness, volume and schema on the lineage graph.
• Lineage history: Store data about lineage versions and changes.
• Column level lineage: Add column-level granularity.
• Full lineage: Integrate with downstream and upstream tools to create a full graph.

Quick start

pip install elementary-lineage

# The tool is named edl (Elementary Data Lineage),
# run it to validate the installation:
edl --help

Now we need a connection file in a simple YAML called profiles.yml. Here is a template to create a Snowflake connection profile.
For further instructions go to our quickstart page.

If you use dbt, you can start right away by running this command with the path to your profiles.yml and the relevant profile name:

edl -d ~/profiles_dir -p <profile_name>

If you like what we are building, support us with .

Documentation

Our full documentation is available here.

Community & Support

For additional information and help, you can use one of these channels:

• Slack (live chat with the team, feature requests, community support, discussions, etc.)
• 📧 Contact us directly at [e-mail]

Integrations

• Snowflake
• BigQuery
• Redshift

Ask us for integrations on Slack or as a Github issue.

License

Elementary lineage is licensed under Apache License 2.0. See the LICENSE file for licensing information.