Skip to main content

Data monitoring and lineage

Project description

Logo

License Downloads GitHub last commit

Monitor your data warehouse in minutes, and be the first to know of data issues. Gain immediate visibility, monitor your data and operation, detect data issues, and understand the impact and root cause.

  • Data monitoring - Set up monitoring for your warehouse in minutes, collect data quality metrics and detect data issues before your users do.

  • Data lineage made simple, reliable, and automated - Tracing the actual upstream & downstream dependencies in the data warehouse.

  • Operational monitoring - Collect dbt artifacts, run and test results as part of your runs. Monitor the operations of your data stack, and centralize metadata easily.

  • Slack alerts - Get notifications on data issues, schema changes, jobs and tests failures.

Join our Slack to learn more on Elementary.

:star: If you like what we are building, support us with a :star:

Demo & sandbox

Try out our live lineage sandbox here.

Data monitoring - Demo coming soon.

Slack

Documentation

How to start? Quickstart.

Questions about the configuration? Configuration FAQ here.

Curious to learn about the different modules? Modules overview.

Our full documentation is available here.

Quick start

Install & connect

pip install elementary-data

edr --help

Add your data warehouse connection details in a profiles.yml file, see our quickstart page to learn more or use this template here. Yes, if you are a dbt user we use dbt's profiles.yml by default (simply add a new profile called 'elementary').

Data monitoring

Add our dbt package to your project, and configure monitors. Execute it using:

edr monitor

To learn how to continuously monitor your data and get Slack notifications refer data monitoring documentation.

Data lineage

# Generate data lineage graph
edr lineage 

# Filter the graph for a specific table, direction and depth
edr lineage -t +my_table+3

To learn more refer data lineage documentation.

Community & Support

For additional information and help, you can use one of these channels:

  • Slack (Live chat with the team, support, discussions, etc.)
  • GitHub issues (Bug reports, feature requests)
  • Roadmap (Vote for features and add your inputs)
  • Twitter (Updates on new releases and stuff)

Integrations

  • Snowflake - Lineage & monitoring
  • BigQuery - Lineage only
  • Redshift

Ask us for integrations on Slack or as a GitHub issue.

License

Elementary is licensed under Apache License 2.0. See the LICENSE file for licensing information.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

elementary-data-0.2.0.tar.gz (63.4 kB view hashes)

Uploaded Source

Built Distribution

elementary_data-0.2.0-py3-none-any.whl (70.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page