Skip to main content

A light-weight and flexible data validation and testing tool for statistical data objects.

Project description


The Open-source Framework for Validating DataFrame-like Objects

📊 🔎 ✅

Data validation for scientists, engineers, and analysts seeking correctness.


CI Build Documentation Status PyPI version shields.io PyPI license pyOpenSci Project Status: Active – The project has reached a stable, usable state and is being actively developed. Documentation Status codecov PyPI pyversions DOI asv Monthly Downloads Total Downloads Conda Downloads Slack

Pandera is a Union.ai open source project that provides a flexible and expressive API for performing data validation on dataframe-like objects. The goal of Pandera is to make data processing pipelines more readable and robust with statistically typed dataframes.

Install

Pandera supports multiple dataframe libraries, including pandas, polars, pyspark, and more. To validate pandas DataFrames, install Pandera with the pandas extra:

With pip:

pip install 'pandera[pandas]'

With uv:

uv pip install 'pandera[pandas]'

With conda:

conda install -c conda-forge pandera-pandas

Get started

First, create a dataframe:

import pandas as pd
import pandera.pandas as pa

# data to validate
df = pd.DataFrame({
    "column1": [1, 2, 3],
    "column2": [1.1, 1.2, 1.3],
    "column3": ["a", "b", "c"],
})

Validate the data using the object-based API:

# define a schema
schema = pa.DataFrameSchema({
    "column1": pa.Column(int, pa.Check.ge(0)),
    "column2": pa.Column(float, pa.Check.lt(10)),
    "column3": pa.Column(
        str,
        [
            pa.Check.isin([*"abc"]),
            pa.Check(lambda series: series.str.len() == 1),
        ]
    ),
})

print(schema.validate(df))
#    column1  column2 column3
# 0        1      1.1       a
# 1        2      1.2       b
# 2        3      1.3       c

Or validate the data using the class-based API:

# define a schema
class Schema(pa.DataFrameModel):
    column1: int = pa.Field(ge=0)
    column2: float = pa.Field(lt=10)
    column3: str = pa.Field(isin=[*"abc"])

    @pa.check("column3")
    def custom_check(cls, series: pd.Series) -> pd.Series:
        return series.str.len() == 1

print(Schema.validate(df))
#    column1  column2 column3
# 0        1      1.1       a
# 1        2      1.2       b
# 2        3      1.3       c

[!WARNING] Pandera v0.24.0 introduces the pandera.pandas module, which is now the (highly) recommended way of defining DataFrameSchemas and DataFrameModels for pandas data structures like DataFrames. Defining a dataframe schema from the top-level pandera module will produce a FutureWarning:

import pandera as pa

schema = pa.DataFrameSchema({"col": pa.Column(str)})

Update your import to:

import pandera.pandas as pa

And all of the rest of your pandera code should work. Using the top-level pandera module to access DataFrameSchema and the other pandera classes or functions will be deprecated in a future version

Next steps

See the official documentation to learn more.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pandera-0.24.0.tar.gz (513.2 kB view details)

Uploaded Source

Built Distribution

pandera-0.24.0-py3-none-any.whl (267.1 kB view details)

Uploaded Python 3

File details

Details for the file pandera-0.24.0.tar.gz.

File metadata

  • Download URL: pandera-0.24.0.tar.gz
  • Upload date:
  • Size: 513.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for pandera-0.24.0.tar.gz
Algorithm Hash digest
SHA256 154231780643bc73b121bd976b0ada9dcebb3e065c622954fd099dc299cf44bd
MD5 9ce8f542101440ede1bc3400d836b481
BLAKE2b-256 474e9eaf279c32058839361085faf203d728967d0fe1d6229dd989139bdb34e0

See more details on using hashes here.

File details

Details for the file pandera-0.24.0-py3-none-any.whl.

File metadata

  • Download URL: pandera-0.24.0-py3-none-any.whl
  • Upload date:
  • Size: 267.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for pandera-0.24.0-py3-none-any.whl
Algorithm Hash digest
SHA256 3b7de575b43a4aa03a1561802be4ee0c7471e49153684fe327f7bbea0724b02b
MD5 3c68db038bb7f2327cea2edab0fb2d9b
BLAKE2b-256 e94dec550fbb5fe09a7c0a93a3f9b2d87446f728c1ef052f786b195eb4620def

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page