Skip to main content

PiperRider CLI

Project description

PipeRider

PipeRider: Data Reliability Automated

What is PipeRider?

PipeRider is an open-source toolkit for detecting data issues across pipelines that work with CI systems for continuous data quality assessment. PipeRider makes it easy for teams to have visibility about how data are being tested, and ensure errors are caught before they cause outages of downstream data applications like business intelligence dashboards or ML systems.

Why PipeRider?

Ensuring consistent quality of data used to be difficult. Missing values, schema changes and data drift (to name just a few) could be introduced to your data at any time. Without effective data quality tools, these errors will affect downstream operations and result in countless lost hours of debugging and missed revenue opportunities from unexpected downtime. PipeRider allows you to define the shape of your data once, and then use the data checking functionality to alert you to changes in your data quality.

Read more about why we created PipeRider.

ci-tests release pipy python downloads license InfuseAI Discord Invite

Learn More

PipeRider Resources Description
Documentation PipeRider Main Doc Site
Sample_Project Sample Project with with sqlite
dbt_Sample_Project Sample Project with dbt
Roadmap PipeRider Roadmap

Key Features

Instant quality assessment in HTML report

Example of single run report

Report Comparison

Example of comparison report

Extensible custom assertions

Refer to custom assertions

Works with existing dbt projects

Automatic Test Recommendations (Coming Soon)

Getting started

Install PipeRider

pip install piperider

By default, PipeRider supports built-in SQLite connector, extra connectors are available:

connectors install
snowflake pip install 'piperider[snowflake]'
postgres pip install 'piperider[postgres]'

Use comma to install multiple connectors in one line:

pip install 'piperider[postgres,snowflake]'

Attach PipeRider to a dbt project

piperider_init

This command creates /.piperider under a dbt project root and generates necessary configurations.

Scan data quality from models

piperider_run

This command scans the models from data sources and creates assessment results in /.piperider/output

Generate reports

piperider_report

generate a static HTML report under the current path.

Generate comparison view

piperider_compare

The generated report in HTML will be placed in the path shown in the console.

Get involved

Contributions

We welcome contributions. See the Set up dev environment and the Contributing guildline to get started.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

piperider-nightly-0.4.0.20220706.tar.gz (627.5 kB view details)

Uploaded Source

Built Distribution

piperider_nightly-0.4.0.20220706-py3-none-any.whl (684.5 kB view details)

Uploaded Python 3

File details

Details for the file piperider-nightly-0.4.0.20220706.tar.gz.

File metadata

File hashes

Hashes for piperider-nightly-0.4.0.20220706.tar.gz
Algorithm Hash digest
SHA256 b7317555370130545169e3cb5138647d0aeed59b4215a7be9125c0a1373e7800
MD5 7d5ca9b63bcb23dc36a3f95bfd6dc0ea
BLAKE2b-256 1949402564c3cc193d58d2ab6735849c6673230f1a04b6e46f0c018ed548986c

See more details on using hashes here.

File details

Details for the file piperider_nightly-0.4.0.20220706-py3-none-any.whl.

File metadata

File hashes

Hashes for piperider_nightly-0.4.0.20220706-py3-none-any.whl
Algorithm Hash digest
SHA256 11994d10963c164ec96f4ed1adef9815eb4a8924524754cb00b4dfd5b78c1be0
MD5 53ec8cdd7e07fde1edac31dd42d0be24
BLAKE2b-256 40b2b4a71fffe2b76ce2c34bdc4d144c9327eeabcfe15c14a5160abb5773de99

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page