dagster-soda
dagster-soda integrates Soda Core data quality checks with Dagster. It provides the SodaScanComponent, a Dagster component that runs Soda Core scans and maps SodaCL check results to Dagster asset checks.
Installation
pip install dagster-soda
Note: dagster-soda requires soda-core 3.x (the soda.scan API). It pins soda-core>=3.0,<4 by default.
Usage
Component: SodaScanComponent
Configure a SodaScanComponent in your Dagster project to:
- Point at SodaCL YAML check files and a Soda
configuration.yml - Map Soda dataset names to Dagster asset keys
- Run scans and report pass/fail as Dagster asset check results
Scaffolding with the CLI
Use the Dagster CLI to scaffold a new Soda scan component in your project (requires dagster-dg-cli):
dg scaffold defs dagster_soda.SodaScanComponent <path>
Example (scaffold into a folder named soda_checks under your defs directory):
dg scaffold defs dagster_soda.SodaScanComponent soda_checks
This generates:
- A defs.yaml with default attributes (
checks_paths,configuration_path,data_source_name,asset_key_map) - A checks.yml template with example SodaCL (e.g.
checks for my_table: - row_count > 0)
Edit the generated files to match your data source and checks, then load your definitions as usual.
Minimal defs.yaml example
type: dagster_soda.SodaScanComponent
attributes:
checks_paths:
- checks.yml
configuration_path: configuration.yml
data_source_name: my_datasource
asset_key_map:
my_table: my_table
Documentation
The docs for dagster-soda can be found here.
1 maintainer
Release files for dagster-soda 0.29.19
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| dagster_soda-0.29.19.tar.gz | 201.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dagster_soda-0.29.19-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 213.7 kB
Release files / dagster_soda-0.29.19.tar.gz
| Download URL | dagster_soda-0.29.19.tar.gz |
|---|---|
| Size | 201.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f2d098b15c3e52ce669e5fbe86009b006881f716450f9f5cd7568c123ef52cfe
|
|
BLAKE2b-256 checksum How to use checksums |
3619ae669f1ad284156230b2db4f5e7f94c83971d6cb348f05cb991554510b3a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.14
|
Release files / dagster_soda-0.29.19-py3-none-any.whl
| Download URL | dagster_soda-0.29.19-py3-none-any.whl |
|---|---|
| Size | 12.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0915612d6a6acbbc02ac1ba6bcf1ee49e85c3b996d430f37f5a96d2129db4050
|
|
BLAKE2b-256 checksum How to use checksums |
ab55956d7e2d81505020c47aa2ba4f2890dcf54534313ae53e6ae26e4be5edac
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.14
|
