Skip to main content

dataframely — A declarative, 🐻‍❄️-native data frame validation library

CI Nightly CI conda-forge pypi-version python-version codecov

🗂 Table of Contents

📖 Introduction

Dataframely is a Python package to validate the schema and content of polars data frames. Its purpose is to make data pipelines more robust by ensuring that data meets expectations and more readable by adding schema information to data frame type hints.

💿 Installation

You can install dataframely using your favorite package manager, e.g., pixi or pip:

pixi add dataframely
pip install dataframely

🎯 Usage

Defining a data frame schema

import dataframely as dy
import polars as pl

class HouseSchema(dy.Schema):
    zip_code = dy.String(nullable=False, min_length=3)
    num_bedrooms = dy.UInt8(nullable=False)
    num_bathrooms = dy.UInt8(nullable=False)
    price = dy.Float64(nullable=False)

    @dy.rule()
    def reasonable_bathroom_to_bedroom_ratio(cls) -> pl.Expr:
        ratio = pl.col("num_bathrooms") / pl.col("num_bedrooms")
        return (ratio >= 1 / 3) & (ratio <= 3)

    @dy.rule(group_by=["zip_code"])
    def minimum_zip_code_count(cls) -> pl.Expr:
        return pl.len() >= 2

Validating data against schema

import polars as pl

df = pl.DataFrame({
    "zip_code": ["01234", "01234", "1", "213", "123", "213"],
    "num_bedrooms": [2, 2, 1, None, None, 2],
    "num_bathrooms": [1, 2, 1, 1, 0, 8],
    "price": [100_000, 110_000, 50_000, 80_000, 60_000, 160_000]
})

# Validate the data and cast columns to expected types
validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)

See more advanced usage examples in the documentation.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataframely-2.13.2.tar.gz (460.9 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

dataframely-2.13.2-cp310-abi3-win_amd64.whl (8.6 MB view details)

Uploaded CPython 3.10+Windows x86-64

dataframely-2.13.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

dataframely-2.13.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (8.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

dataframely-2.13.2-cp310-abi3-macosx_11_0_arm64.whl (7.6 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

dataframely-2.13.2-cp310-abi3-macosx_10_12_x86_64.whl (8.2 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file dataframely-2.13.2.tar.gz.

File metadata

  • Download URL: dataframely-2.13.2.tar.gz
  • Upload date:
  • Size: 460.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dataframely-2.13.2.tar.gz
Algorithm Hash digest
SHA256 9cb7a0033dfc35d185973bb4f0305c062cc6dc050844ad28e7274a15a9a849ce
MD5 d49d86c248a9a0ba3ecbc13e160c3f10
BLAKE2b-256 38db1c2e939ed87b192862c3b52a20d3012cdca6f3432d1ca8a919834ecc5b35

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.2.tar.gz:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.2-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: dataframely-2.13.2-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 8.6 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dataframely-2.13.2-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 5d38ea792dc7b5e8039728ffd34e45e0175671fdaba7c19d7a95a8db926fc126
MD5 b686f0e16e413461d40adc89d3deb2fe
BLAKE2b-256 3e5e41c6a7d3973a1c7cb960347c7f2252a128abb93a70622cf2def8e25e5ffb

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.2-cp310-abi3-win_amd64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 aca76460ed5e1309229fe91bb17ef1592aeaf0085304656de6712abd39dbc3c9
MD5 9f905d8adcc6582f9fa7616bb169dcd2
BLAKE2b-256 8b4d8605d9d3c3f858afc61a58b1ff0809cc9799b37999ae7e8eb6f822ac8055

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 4b61c101df384be7d4b8735bd3eea8f1637e32e3e08a2e7d84478ad0b3236694
MD5 04854b2fa05d342edd754a91d404f3c0
BLAKE2b-256 484d9c675616530d983f6b99481fc9ae689263277a3aea64bddf4bf5b21fde49

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.2-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.2-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4ac5b8c140606aef64994252f7c7203c61038cf21b6faa6f5cb39e3164b1f508
MD5 7cc96b99d6c16e292d1aad8ca221d5c2
BLAKE2b-256 f514cdfbb353271f9d68359976b86c34840aaafb7a1aaf3c808edae8a2cd844b

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.2-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dataframely-2.13.2-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for dataframely-2.13.2-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 0fb1b183a86801e31007f4ae15500ad44cc30281e9b9c36ce84e5c5c45913b03
MD5 e23618977de2fbecaff887b829bad9a9
BLAKE2b-256 1791b904a4652fd5534838bd1a028ee06006546ecf6b41ef3b36df41cd9237f6

See more details on using hashes here.

Provenance

The following attestation bundles were made for dataframely-2.13.2-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on Quantco/dataframely

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page