Skip to main content

dataframely — A declarative, 🐻‍❄️-native data frame validation library

CI Nightly CI conda-forge pypi-version python-version codecov

🗂 Table of Contents

📖 Introduction

Dataframely is a Python package to validate the schema and content of polars data frames. Its purpose is to make data pipelines more robust by ensuring that data meets expectations and more readable by adding schema information to data frame type hints.

💿 Installation

You can install dataframely using your favorite package manager, e.g., pixi or pip:

pixi add dataframely
pip install dataframely

🎯 Usage

Defining a data frame schema

import dataframely as dy
import polars as pl

class HouseSchema(dy.Schema):
    zip_code = dy.String(nullable=False, min_length=3)
    num_bedrooms = dy.UInt8(nullable=False)
    num_bathrooms = dy.UInt8(nullable=False)
    price = dy.Float64(nullable=False)

    @dy.rule()
    def reasonable_bathroom_to_bedroom_ratio(cls) -> pl.Expr:
        ratio = pl.col("num_bathrooms") / pl.col("num_bedrooms")
        return (ratio >= 1 / 3) & (ratio <= 3)

    @dy.rule()
    def minimum_zip_code_count(cls) -> pl.Expr:
        return (pl.len() >= 2).over("zip_code")

Validating data against schema

import polars as pl

df = pl.DataFrame({
    "zip_code": ["01234", "01234", "1", "213", "123", "213"],
    "num_bedrooms": [2, 2, 1, None, None, 2],
    "num_bathrooms": [1, 2, 1, 1, 0, 8],
    "price": [100_000, 110_000, 50_000, 80_000, 60_000, 160_000]
})

# Validate the data and cast columns to expected types
validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)

See more advanced usage examples in the documentation.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataframely-3.0.0.tar.gz (428.4 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dataframely-3.0.0-cp310-abi3-win_amd64.whl (8.8 MB view details)

Uploaded CPython 3.10+Windows x86-64

dataframely-3.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

dataframely-3.0.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (8.2 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

dataframely-3.0.0-cp310-abi3-macosx_11_0_arm64.whl (7.8 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

dataframely-3.0.0-cp310-abi3-macosx_10_12_x86_64.whl (8.4 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file dataframely-3.0.0.tar.gz.

File metadata

  • Download URL: dataframely-3.0.0.tar.gz
  • Upload date:
  • Size: 428.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for dataframely-3.0.0.tar.gz
Algorithm Hash digest
SHA256 122ddcf9926f5b9986e2e414f5d1a504e2140f0854c9a6f6199cf437e4b25415
MD5 762aedbf45acab4a8cc7bb4c362088b4
BLAKE2b-256 b2a04bf2f61a58eb2d883aed94d29ce67e976c130050965eeb3c788bca3ee30b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-3.0.0.tar.gz:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-3.0.0-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: dataframely-3.0.0-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 8.8 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for dataframely-3.0.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 7dccc40cc5f0018975bfb4bfde658a4d00e9137e92068b8e6d6866edee8e3cf4
MD5 8bee76198ab3bb8d7766d1f33add1ac7
BLAKE2b-256 76e4c60edba1e888d322d9226ecd311d3577948fbdc5a79bed002109f590bf6c

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-3.0.0-cp310-abi3-win_amd64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-3.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-3.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 13ab9c2cc600583b500086feaded312466d60e333039786ee306d942602425ae
MD5 9898a63591886d9ac07c161ebe77cd24
BLAKE2b-256 819f271d6d3b91c63851464dd4cd821c54b74dbe193e737ce4f3907ca03b1e91

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-3.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-3.0.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for dataframely-3.0.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 5d7565a3e72dc9022f33a5eb0cc440679676415f44c90cc22b54e32fa924f33f
MD5 b2f9f181adc2cf058fe285e7a0b8a736
BLAKE2b-256 4502b4fa6de5f9022fc44520c0b11ef8bd444a38d40f9706ccb0d515d21613b8

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-3.0.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-3.0.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dataframely-3.0.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 6ef9de04d24f40036547bf633a13fd8dc4e01dfbaa2fb3df5b96251b0e2ee26f
MD5 28b50ab4031a85b58eea3f7fa80c12e0
BLAKE2b-256 ff79381fff5a369dc5d374c6ce203cdaeb14b1cfe260d301355b5e22912d43a8

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-3.0.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-3.0.0-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-3.0.0-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 ea4305be0856c08e199ac1cbfaf662c4e7af110e29353a341df960c396e20238
MD5 5a35e0ed1234b54c0fad84b4e137210f
BLAKE2b-256 dedc9183dba97f19802b9d9a5a8b14ae4189945b9d70eba33256fc33487cd978

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-3.0.0-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page