elementary-data 0.2.0

Data monitoring and lineage

Monitor your data warehouse in minutes, and be the first to know of data issues. Gain immediate visibility, monitor your data and operation, detect data issues, and understand the impact and root cause.

• Data monitoring - Set up monitoring for your warehouse in minutes, collect data quality metrics and detect data issues before your users do.

• Data lineage made simple, reliable, and automated - Tracing the actual upstream & downstream dependencies in the data warehouse.

• Operational monitoring - Collect dbt artifacts, run and test results as part of your runs. Monitor the operations of your data stack, and centralize metadata easily.

• Slack alerts - Get notifications on data issues, schema changes, jobs and tests failures.

Join our Slack to learn more on Elementary.

:star: If you like what we are building, support us with a :star:

Demo & sandbox

Try out our live lineage sandbox here.

Data monitoring - Demo coming soon.

Documentation

How to start? Quickstart.

Questions about the configuration? Configuration FAQ here.

Curious to learn about the different modules? Modules overview.

Our full documentation is available here.

Quick start

Install & connect

pip install elementary-data

edr --help

Add your data warehouse connection details in a profiles.yml file, see our quickstart page to learn more or use this template here. Yes, if you are a dbt user we use dbt's profiles.yml by default (simply add a new profile called 'elementary').

Data monitoring

Add our dbt package to your project, and configure monitors. Execute it using:

edr monitor

To learn how to continuously monitor your data and get Slack notifications refer data monitoring documentation.

Data lineage

# Generate data lineage graph
edr lineage 

# Filter the graph for a specific table, direction and depth
edr lineage -t +my_table+3

To learn more refer data lineage documentation.

Community & Support

For additional information and help, you can use one of these channels:

• Slack (Live chat with the team, support, discussions, etc.)
• GitHub issues (Bug reports, feature requests)
• Roadmap (Vote for features and add your inputs)
• Twitter (Updates on new releases and stuff)

Integrations

• Snowflake - Lineage & monitoring
• BigQuery - Lineage only
• Redshift

Ask us for integrations on Slack or as a GitHub issue.

License

Elementary is licensed under Apache License 2.0. See the LICENSE file for licensing information.