Skip to main content

dataframely — A declarative, 🐻‍❄️-native data frame validation library

CI Nightly CI conda-forge pypi-version python-version codecov

🗂 Table of Contents

📖 Introduction

Dataframely is a Python package to validate the schema and content of polars data frames. Its purpose is to make data pipelines more robust by ensuring that data meets expectations and more readable by adding schema information to data frame type hints.

💿 Installation

You can install dataframely using your favorite package manager, e.g., pixi or pip:

pixi add dataframely
pip install dataframely

🎯 Usage

Defining a data frame schema

import dataframely as dy
import polars as pl

class HouseSchema(dy.Schema):
    zip_code = dy.String(nullable=False, min_length=3)
    num_bedrooms = dy.UInt8(nullable=False)
    num_bathrooms = dy.UInt8(nullable=False)
    price = dy.Float64(nullable=False)

    @dy.rule()
    def reasonable_bathroom_to_bedroom_ratio(cls) -> pl.Expr:
        ratio = pl.col("num_bathrooms") / pl.col("num_bedrooms")
        return (ratio >= 1 / 3) & (ratio <= 3)

    @dy.rule(group_by=["zip_code"])
    def minimum_zip_code_count(cls) -> pl.Expr:
        return pl.len() >= 2

Validating data against schema

import polars as pl

df = pl.DataFrame({
    "zip_code": ["01234", "01234", "1", "213", "123", "213"],
    "num_bedrooms": [2, 2, 1, None, None, 2],
    "num_bathrooms": [1, 2, 1, 1, 0, 8],
    "price": [100_000, 110_000, 50_000, 80_000, 60_000, 160_000]
})

# Validate the data and cast columns to expected types
validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)

See more advanced usage examples in the documentation.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataframely-2.13.1.tar.gz (459.6 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dataframely-2.13.1-cp310-abi3-win_amd64.whl (8.6 MB view details)

Uploaded CPython 3.10+Windows x86-64

dataframely-2.13.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

dataframely-2.13.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (8.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

dataframely-2.13.1-cp310-abi3-macosx_11_0_arm64.whl (7.6 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

dataframely-2.13.1-cp310-abi3-macosx_10_12_x86_64.whl (8.2 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file dataframely-2.13.1.tar.gz.

File metadata

  • Download URL: dataframely-2.13.1.tar.gz
  • Upload date:
  • Size: 459.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dataframely-2.13.1.tar.gz
Algorithm Hash digest
SHA256 2ed77e2789b9049ac94b531432691d5f1a2726cee76332b1a5ba61d73916d430
MD5 3d4f807dac21782a9ae43dead852312a
BLAKE2b-256 7ac5d54e038e16ea856fe9c3b76f44cd4064d8ab1d692915807982fd66492639

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.1.tar.gz:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.1-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: dataframely-2.13.1-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 8.6 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dataframely-2.13.1-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 e86bf0fa03af1450da3c42a410d2acd322109de0eca596ffd7109934ba957896
MD5 c4b3973ee54e4f4be08b12fcf15e2460
BLAKE2b-256 9bfc0312c7f875c99e5a3ad19991a8faf3bd32624ec681dfbf4916eeffb70a29

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.1-cp310-abi3-win_amd64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 812dd8000a8c5ffa7d468d5a6592c20f8b7ad65c8c3385b2a1b52d71f24bbaa7
MD5 20ad778e1cbbeb0b4aed82d43dc98df2
BLAKE2b-256 c2f4d77b7c025c521f0a3f6d8fed0dd0294a662db2f7d13efdfe82552c6c3b82

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ab83500dda4095772f8b57de42a259a608aff37bd4fc2be35af8f5b8c79f9737
MD5 59484c6718c9f4177a4dbb6fea979f26
BLAKE2b-256 52c9f24553f1184b18611d7ffe8e35f1dcb30c0f5cbd6ccce04aecd539be3759

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.1-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.1-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 94f031c787c9216c156cd85b5053639b2bd0fc624fd485e1fd56b4b35bd2a416
MD5 5ffd91fa224fe0332f1f65d6ac1d2959
BLAKE2b-256 6811530dbb48093994f1f0a2d042937523d50df720ffad8d973c84798021e376

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.1-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.1-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.1-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 5bc5c063fb47cabf84aef3655650456ac1fafd7d267b83d8ea2de54cb9b863ce
MD5 f32282e414548c55669245c77d367560
BLAKE2b-256 89b39ec8f1f6d234a77ef959a7c902f3cdff1911b27dede40d6efa94de5280df

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.1-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page