Skip to main content

PiperRider CLI

Project description

PipeRider: Data Reliability Automated

ci-tests release pipy python downloads license InfuseAI Discord Invite

PipeRider

What's PipeRider?

PipeRider is a light-weight data quality tool so you can be confident of your data without writing tests for every single pipeline.

We're in an early stage, so let us know if you have any questions, feedback, or need help installing PipeRider! :heart:

Key Features

Made for...

For data engineers

  • :zap: 2 min install & set-up
  • :relieved: non-intrusive & open-source: install and use locally
  • :moneybag: fast & cheap: 10M rows, 8 columns takes only 16s to profile

For data analysts

  • :bowtie: never waste time on analyzing wrong data: collects various metadata metrics such as freshness, uniqueness, distribution... check all metrics
  • :speech_balloon: communicate easily your data expectations by showing the report

Live Demo

Click here or on image to interact

Getting started

Get started quickly below, or go to the docs

Install PipeRider

pip install piperider

By default, PipeRider supports built-in SQLite connector, extra connectors are available:

connectors install supported since
snowflake pip install 'piperider[snowflake]'
postgres pip install 'piperider[postgres]'
bigquery pip install 'piperider[bigquery]' PipeRider v0.7.0
redshift pip install 'piperider[redshift]' PipeRider v0.7.0
parquet pip install 'piperider[parquet]' PipeRider v0.8.0
csv pip install 'piperider[csv]' PipeRider v0.8.0
duckdb pip install 'piperider[duckdb]' PipeRider v0.8.0

Use comma to install multiple connectors in one line:

pip install 'piperider[postgres,snowflake]'

You can follow the quick start guide to learn more about PipeRider.

Attach PipeRider to a dbt project

piperider init creates /.piperider under a dbt project root and generates necessary configurations.

Scan data quality from models

piperider run runs profiling, and tests against assertions, and will render an HTML report /.piperider/outputs

You can do a run for a specific table using piperider run --table $TABLENAME You can specifying the output location of a report using piperider generate-report -o $PATHNAME or specifying it in the config file.

All details on generating reports

Generate comparison view

You can use piperider compare-report to compare 2 reports.

Learn More

PipeRider Resources Description
Documentation PipeRider Main Doc Site
Sample_Project Sample Project with with sqlite
dbt_Sample_Project Sample Project with dbt
Roadmap PipeRider Roadmap
Blog How we got started

Get involved

Support :heart:

If you like what we are building, support us! Give us a :star: or get in touch. We'd love your feedback! Send us a message on piperider.io, join our Discord, or report an issue on GitHub

Contributions

We welcome contributions. See the Set up dev environment and the Contributing guildline to get started.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

piperider-0.9.0rc1.tar.gz (3.6 MB view hashes)

Uploaded Source

Built Distribution

piperider-0.9.0rc1-py3-none-any.whl (3.7 MB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page