Presenting data lineage based on your data warehouse query history
Project description
Elementary is an open-source data reliability framework for modern data teams. The first module of the framework is data lineage.
Data lineage made simple, reliable, and automated.
Open-source tool to effortlessly track the flow of data, understand dependencies and analyze impact.
Features
- Visualization: In browser visual representation of the data lineage graph.
- DWH lineage: Effortlessly map data flow in the data warehouse.
- Accuracy: Reflects the actual state in the DWH.
- Plug-and-play: No need for code changes.
Coming soon:
- Monitoring: Present data about freshness, volume and schema on the lineage graph.
- Lineage history: Store data about lineage versions and changes.
- Column level lineage: Add column-level granularity.
- Full lineage: Integrate with downstream and upstream tools to create a full graph.
Quick start
pip install elementary-lineage
# The tool is named edl (Elementary Data Lineage),
# run it to validate the installation:
edl --help
Now we need a connection file in a simple YAML called profiles.yml
.
If you don't use dbt, go to our quickstart page for instructions on how to create it.
If you use dbt, you can start right away by running this command with the path to your profiles.yml
and the relevant profile name:
edl -d ~/.dbt -p <profile_name>
Documentation
Our full documentation is available here.
Community & Support
For additional information and help, you can use one of these channels:
- Slack (live chat with the team, feature requests, community support, discussions, etc.)
- 📧Contact us directly at team@elementary-data.com
Integrations
- Snwoflake
- BigQuery
- Redshift
Ask us for integrations on Slack or as a Github issue.
License
Elementary is licensed under Apache License 2.0. See the LICENSE file for licensing information.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for elementary_lineage-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c8ab5d0749ecffcc58e46b382e263e4d3559cee6fedfb9ba0825460e8c120935 |
|
MD5 | fb319bedd3db10eda7edb26f42723440 |
|
BLAKE2b-256 | c986cb298d1437db758385491c45f6f9de9f4b3f9f67a01522deef507e6c730d |