Skip to main content

dataframely — A declarative, 🐻‍❄️-native data frame validation library

CI Nightly CI conda-forge pypi-version python-version codecov

🗂 Table of Contents

📖 Introduction

Dataframely is a Python package to validate the schema and content of polars data frames. Its purpose is to make data pipelines more robust by ensuring that data meets expectations and more readable by adding schema information to data frame type hints.

💿 Installation

You can install dataframely using your favorite package manager, e.g., pixi or pip:

pixi add dataframely
pip install dataframely

🎯 Usage

Defining a data frame schema

import dataframely as dy
import polars as pl

class HouseSchema(dy.Schema):
    zip_code = dy.String(nullable=False, min_length=3)
    num_bedrooms = dy.UInt8(nullable=False)
    num_bathrooms = dy.UInt8(nullable=False)
    price = dy.Float64(nullable=False)

    @dy.rule()
    def reasonable_bathroom_to_bedroom_ratio(cls) -> pl.Expr:
        ratio = pl.col("num_bathrooms") / pl.col("num_bedrooms")
        return (ratio >= 1 / 3) & (ratio <= 3)

    @dy.rule(group_by=["zip_code"])
    def minimum_zip_code_count(cls) -> pl.Expr:
        return pl.len() >= 2

Validating data against schema

import polars as pl

df = pl.DataFrame({
    "zip_code": ["01234", "01234", "1", "213", "123", "213"],
    "num_bedrooms": [2, 2, 1, None, None, 2],
    "num_bathrooms": [1, 2, 1, 1, 0, 8],
    "price": [100_000, 110_000, 50_000, 80_000, 60_000, 160_000]
})

# Validate the data and cast columns to expected types
validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)

See more advanced usage examples in the documentation.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataframely-2.13.0.tar.gz (457.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dataframely-2.13.0-cp310-abi3-win_amd64.whl (8.6 MB view details)

Uploaded CPython 3.10+Windows x86-64

dataframely-2.13.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

dataframely-2.13.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (8.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

dataframely-2.13.0-cp310-abi3-macosx_11_0_arm64.whl (7.6 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

dataframely-2.13.0-cp310-abi3-macosx_10_12_x86_64.whl (8.2 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file dataframely-2.13.0.tar.gz.

File metadata

  • Download URL: dataframely-2.13.0.tar.gz
  • Upload date:
  • Size: 457.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dataframely-2.13.0.tar.gz
Algorithm Hash digest
SHA256 481bfb699e9e398dd271c51ff1a96c294fbdd01bdacdbe37de0dfbf9b7b47c62
MD5 835997443f71266e3e7886c6dd9f60bb
BLAKE2b-256 881a70a7f4a57da4e50a1604870989f03e65bda39941dbffd127bd766834ddbf

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.0.tar.gz:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.0-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 71d14342a31ce557aea7f18c0bce626222a1b838e198b7efeea3a9bf10e5c8ad
MD5 8d0c91098b9925fcbfdb1acaf005e54c
BLAKE2b-256 7da548f5130129ec7cf7fd8aa0d798f493819b0ff56edd3ab769647e04d7aa27

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.0-cp310-abi3-win_amd64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4b88f6d996712b511b567bf7cc75f0d56565a3e931aa933e42fb9381d9e228b1
MD5 ddc0c6499ae06052277d2237439a9d74
BLAKE2b-256 2407611ad7e03a955f08325d2452420f137f2b9d01d6d6a76700fd7906d0d49b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 d4915f4c0cf39eedb50ae1f1d5f20748c498b7e7179bb83dbb51b102984a2ad5
MD5 7490f4c4fbae5383c70948e7d33b5403
BLAKE2b-256 1ba9d790be09d03a277e2ded4da2de54597b69dd4f839772783c883e987e0b02

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 22293616514bdfdf649ef12cf2c760f088fdeb156f9caa54d935584926cbfc1c
MD5 4833b63d5dfafcecccd177fecf16c4cc
BLAKE2b-256 b6a8f09991ae994918985e5ec1908e13d4f534cfb2eda9b58146ec77e6fda589

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.0-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.0-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 9df6dc870f5a1b7106a87b965d24dda06ca13cf2741e8e1bbfe5cf5f6c8dbcf4
MD5 9d2dd9cb4b050c2a4dbccc4676c654c8
BLAKE2b-256 4b5b2770d265cf89a18b2a983fa0e4fb935b2a356bb3d87f41d5535aa2d5dadd

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.0-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page