dagster-dataframely

Dataframely plugin for Dagster.

[!WARNING] Pre-release placeholder. This package is under active design and ships no functionality yet. The name is reserved on PyPI while the design spec is finalized. Do not depend on it. Follow issue #1 for progress.

What it will do

dataframely declares schemas and validation rules for polars data frames. Dagster has first-class surfaces for data contracts: column schema metadata and asset checks. This package connects the two:

  • Asset checks derived from your schema. Each dataframely rule becomes a Dagster asset check, so per-rule pass/fail history shows up in the catalog without hand-writing checks.
  • Column schema metadata. Your schema populates Dagster's Columns tab automatically.
  • Quarantine, opt-in. Declare a second output and failing rows are routed there with per-rule attribution instead of failing the run.
  • Storage in the box. DataframelyParquetIOManager writes .parquet to a local directory or to s3://, gs:// and az://, and it is the supported path.

Dependencies are dagster and dataframely (plus polars) only. The IO manager imports universal-pathlib and pydantic directly, so both are declared too. Both already ship with dagster, so nothing new lands in your environment.

Installation

uv add dagster-dataframely

Requires Python 3.12+.

License

Apache-2.0

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ozanozbeker from gravatar.com ozanozbeker

Credits

Author/Maintainer: Ozan Ozbeker

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.12

Tags

dagster data data-quality dataframely polars validation

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dagster_dataframely-0.0.1.tar.gz (19.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dagster_dataframely-0.0.1-py3-none-any.whl (24.1 kB view details)

Uploaded Python 3

File details

Details for the file dagster_dataframely-0.0.1.tar.gz.

File metadata

  • Download URL: dagster_dataframely-0.0.1.tar.gz
  • Upload date:
  • Size: 19.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for dagster_dataframely-0.0.1.tar.gz
Algorithm Hash digest
SHA256 f4bf34b1d825d1cd6d70b71626c315192f649a4385321fa0d6d94b75c55dc285
MD5 6a9a7daac445f3546f823f6c2b46e648
BLAKE2b-256 03573d3170186f0ca713e012440902ce5724b9a6ba8a2c3d19e519090ab69a9c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dagster_dataframely-0.0.1.tar.gz:

Publisher: release.yml on ozanozbeker/dagster-dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dagster_dataframely-0.0.1-py3-none-any.whl.

File metadata

File hashes

Hashes for dagster_dataframely-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 e8c00538d3584b2b35c6c1eb9b9c3f6078e7c308645f4f41eae756a5850a9c92
MD5 66cf143a657483a3e7ebdecb8aab826f
BLAKE2b-256 e95ac30cf77a524e80b73d699792f25f15daf3db8eca36d420975299daea7d5e

See more details on using hashes here.

Provenance

The following attestation bundles were made for dagster_dataframely-0.0.1-py3-none-any.whl:

Publisher: release.yml on ozanozbeker/dagster-dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page