This release is a pre-release and may not be stable for production use.
PipeRider: Data Reliability Automated
What is PipeRider?
PipeRider is an open-source toolkit for detecting data issues across pipelines that work with CI systems for continuous data quality assessment. PipeRider makes it easy for teams to have visibility about how data are being tested, and ensure errors are caught before they cause outages of downstream data applications like business intelligence dashboards or ML systems.
Why PipeRider?
Ensuring consistent quality of data used to be difficult. Missing values, schema changes and data drift (to name just a few) could be introduced to your data at any time. Without effective data quality tools, these errors will affect downstream operations and result in countless lost hours of debugging and missed revenue opportunities from unexpected downtime. PipeRider allows you to define the shape of your data once, and then use the data checking functionality to alert you to changes in your data quality.
Learn More
| PipeRider Resources | Description |
|---|---|
| Documentation | PipeRider Main Doc Site |
| Sample_Project | Sample Project with with sqlite |
| dbt_Sample_Project | Sample Project with dbt |
| Roadmap | PipeRider Roadmap |
Key Features
Instant quality assessment in HTML report
Example of single run report
Report Comparison
Example of comparison report
Extensible custom assertions
Refer to custom assertions
Works with existing dbt projects
Automatic Test Recommendations (Coming Soon)
Getting started
Install PipeRider
pip install piperider
By default, PipeRider supports built-in SQLite connector, extra connectors are available:
| connectors | install |
|---|---|
| snowflake | pip install 'piperider[snowflake]' |
| postgres | pip install 'piperider[postgres]' |
Use comma to install multiple connectors in one line:
pip install 'piperider[postgres,snowflake]'
Attach PipeRider to a dbt project
This command creates /.piperider under a dbt project root and generates necessary configurations.
Scan data quality from models
This command scans the models from data sources and creates assessment results in /.piperider/output
Generate reports
generate a static HTML report under the current path.
Generate comparison view
The generated report in HTML will be placed in the path shown in the console.
Get involved
Contributions
We welcome contributions. See the Set up dev environment and the Contributing guildline to get started.
Metadata
Release files for piperider 0.3.0a269911
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| piperider-0.3.0a269911.tar.gz | 611.3 kB | Without provenance | Details for piperider-0.3.0a269911.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| piperider-0.3.0a269911-py3-none-any.whl | Python 3 | none | any | Details for piperider-0.3.0a269911-py3-none-any.whl |
Total release size: 1.3 MB
Release files / piperider-0.3.0a269911.tar.gz
| Download URL | piperider-0.3.0a269911.tar.gz |
|---|---|
| Size | 611.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b7d9ef0943aa978ec5067a355968c9eb536d822b6d01a0cdd9d491ccc1cc0e0d
|
|
BLAKE2b-256 checksum How to use checksums |
e6ecf7f6ee3ba508c98be3fb959219f39577d1ffe33fe05a616c93f8b990e66d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.8.13
|
Release files / piperider-0.3.0a269911-py3-none-any.whl
| Download URL | piperider-0.3.0a269911-py3-none-any.whl |
|---|---|
| Size | 664.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bd5ffa1f0e6b41502d49259fea221af90acf3d325153c0d78725b0a22e16a3d7
|
|
BLAKE2b-256 checksum How to use checksums |
901ba49c7aa9bc752492544f98481e8f8b8c66ab6273dc1f586ab6edda946822
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.8.13
|
Security for piperider 0.3.0a269911
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.