Skip to main content

dataframely — A declarative, 🐻‍❄️-native data frame validation library

CI Nightly CI conda-forge pypi-version python-version codecov

🗂 Table of Contents

📖 Introduction

Dataframely is a Python package to validate the schema and content of polars data frames. Its purpose is to make data pipelines more robust by ensuring that data meets expectations and more readable by adding schema information to data frame type hints.

💿 Installation

You can install dataframely using your favorite package manager, e.g., pixi or pip:

pixi add dataframely
pip install dataframely

🎯 Usage

Defining a data frame schema

import dataframely as dy
import polars as pl

class HouseSchema(dy.Schema):
    zip_code = dy.String(nullable=False, min_length=3)
    num_bedrooms = dy.UInt8(nullable=False)
    num_bathrooms = dy.UInt8(nullable=False)
    price = dy.Float64(nullable=False)

    @dy.rule()
    def reasonable_bathroom_to_bedroom_ratio(cls) -> pl.Expr:
        ratio = pl.col("num_bathrooms") / pl.col("num_bedrooms")
        return (ratio >= 1 / 3) & (ratio <= 3)

    @dy.rule(group_by=["zip_code"])
    def minimum_zip_code_count(cls) -> pl.Expr:
        return pl.len() >= 2

Validating data against schema

import polars as pl

df = pl.DataFrame({
    "zip_code": ["01234", "01234", "1", "213", "123", "213"],
    "num_bedrooms": [2, 2, 1, None, None, 2],
    "num_bathrooms": [1, 2, 1, 1, 0, 8],
    "price": [100_000, 110_000, 50_000, 80_000, 60_000, 160_000]
})

# Validate the data and cast columns to expected types
validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)

See more advanced usage examples in the documentation.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataframely-2.12.0.tar.gz (456.7 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dataframely-2.12.0-cp310-abi3-win_amd64.whl (8.6 MB view details)

Uploaded CPython 3.10+Windows x86-64

dataframely-2.12.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

dataframely-2.12.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (8.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

dataframely-2.12.0-cp310-abi3-macosx_11_0_arm64.whl (7.6 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

dataframely-2.12.0-cp310-abi3-macosx_10_12_x86_64.whl (8.2 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file dataframely-2.12.0.tar.gz.

File metadata

  • Download URL: dataframely-2.12.0.tar.gz
  • Upload date:
  • Size: 456.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dataframely-2.12.0.tar.gz
Algorithm Hash digest
SHA256 a1262be5f6eb498db50a9bad9c72d87ad0c201a33ab1d9496eb11b2d475d50b6
MD5 e084a94b288362e0cc9a720bb37d228f
BLAKE2b-256 59fc9021efc6e2ee568211d9da6e51e386a52e7ef7fa2195c1b9558d81c06070

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.12.0.tar.gz:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.12.0-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for dataframely-2.12.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 6a616da13d53a7f03b3e52c989e19041f289d7c178545fbb7cf7cb4c1245f98f
MD5 c7a68025af2a5b10fe02c8df8f9c1bc0
BLAKE2b-256 349c4b7d77b510f92ab96cfec35f64c3b8bcce5946915ecb37e89c0fcb9b9bcf

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.12.0-cp310-abi3-win_amd64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.12.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.12.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 dce0064ce2f26d9f1ef373475f67d2cba623da9bc232f9dce7e5585a781c5763
MD5 0a8ee472b0b0a899899ec1ca5ebb2b96
BLAKE2b-256 e55e94f88ef4214acb8505db40cdb4d844ad6c067cde7cdb9fd6cdf5789c63da

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.12.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.12.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for dataframely-2.12.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 88478c2fd65839876d01ab5dd804179381fde6f300dfa0f97cbf0b7e9eda3b06
MD5 832fc79e71dddd7f71d0c67703e5d00f
BLAKE2b-256 6e5a7ff5942095240b1cea8d981783137292a39aae2bb8f9ae7dc43ee11356af

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.12.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.12.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dataframely-2.12.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4ba30043a0b0ec74397593f6fe96c48063d42c1d1010608f0720fe0531cf312d
MD5 e0530210807df6b4dcba14db99e05818
BLAKE2b-256 2e7f3adcc6f1ce0adbe8c5f278dfb8d42ddaac61e4a0990f197a8272821cd29b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.12.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.12.0-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.12.0-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 6b17dc3b4cd19827a22f3044518df7c678c32c27f8859303725043fbc98b54a5
MD5 f8b4235a369873853c235738bca6b063
BLAKE2b-256 bb9649f21464c3c5a0a2e72f38e1d587eb9617e776f3f1b976e3a25d382dbb2d

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.12.0-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page