DQO Data Quality Operations Center
Project description
DQO Data Quality Operations Center
DQO is an DataOps friendly data quality monitoring tool with customizable data quality checks and data quality dashboards. DQO comes with around 100 predefined data quality checks which helps you monitor the quality of your data.
Key features
- Intuitive graphical interface and access via CLI
- Support of a number of different data sources: BigQuery, Snowflake, PostgreSQL, Redshift, SQL Server and MySQL
- ~450 build-in table and column checks with easy customization
- Table and column-level checks which allows writing your own SQL queries
- Daily and monthly date partition testing
- Data segmentation by up to 9 different data streams
- Build-in scheduling
- Calculation of data quality KPIs which can be displayed on multiple built-in data quality dashboards
Installation
To use DQO you need:
- Python version 3.8 or greater (for details see Python's documentation and download sites).
- Ability to install Python packages with pip.
- Installed JDK software (version 17) and set the JAVA_HOME environment variable.
DQO is available on PyPi repository.
-
To install DQO via pip manager just run
=== "Windows"
py -m pip install dqops
=== "MacOS/Linux"
pip install dqops
If you prefer to work with the source code, just clone our GitHub repository https://github.com/dqops/dqo and run
-
Run dqo app to finalize the installation.
=== "Windows"
dqo
=== "MacOS/Linux"./dqo
-
Create DQO
userhome
folder.After installation, you will be asked whether to initialize the DQO
userhome
folder in the default location. Type Y to create the folder.
Theuserhome
folder locally stores data such as sensor readouts and checkout results, as well as data source configurations. You can learn more about data storage here. -
Login to DQO Cloud.
To use DQO features, such as storing data quality definitions and results in the cloud or data quality dashboards, you must create a DQO cloud account.
After creating a userhome folder, you will be asked whether to log in to the DQO cloud. After typing Y, you will be redirected to https://cloud.dqo.ai/registration, where you can create a new account, use Google single sign-on (SSO) or log in if you already have an account.
During the first registration, a unique identification code (API Key) will be generated and automatically retrieved by DQO application. The API Key is now stored in the configuration file.
-
Open the DQO User Interface Console in your browser by CTRL-clicking on the link displayed on the command line (for example http://localhost:8888) or by copying the link.
Documentation
For full documentation with guides and use cases, visit https://dqo.ai/docs
Contact and issues
If you find any issues with the tool, just post it here:
https://github.com/dqops/dqo/issues
or contact us via https://dqo.ai/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dqops-0.2.0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59ab0f4dbeb8b1dda6fe5502045c62cadad2d2c9c72f6eeeabbc563cfa2986af |
|
MD5 | 2d0c62452b6e9cccdb9260c4593588cf |
|
BLAKE2b-256 | 7506af06c6c378e9905d2472a7409758361cbcf5b909ba5cdfda6f5d9d329532 |