Skip to main content

Lightweight, readable, and extensible validation for pandas DataFrames.

Project description

Project Logo

CI codecov Documentation Status PyPI Python

Lightweight, flexible, and intuitive validation for pandas DataFrames.
Define expectations for your data, validate them cleanly, and surface friendly errors or warnings — no configuration files, ever.

🔗 View full documentation →


📦 Installation

pip install framecheck

Try FrameCheck

Open In Colab

Try FrameCheck without installing anything - click the badge above to run an interactive demo notebook in Google Colab.


Main Features

  • Designed for pandas users
  • Simple, fluent API
  • Supports error or warning-level assertions
  • Validates both column-level and DataFrame-level rules
  • No config files, decorators, or boilerplate

Why Should You Validate Your Data Explicitly?

Because during a routine update, a typo like this:

SELECT u.credit_score AS age
FROM users u
JOIN profiles p ON u.id = p.user_id

...fails silently.

Now your age column has values in the 700s.

Your SQL doesn't error, your model still runs — and makes terrible predictions.


🔥 Example: Catch Bad Model Output Before It Hits Production

import pandas as pd
from framecheck import FrameCheck, register_check_function

df = pd.DataFrame({
    'transaction_id': ['TXN1001', 'TXN1002', 'TXN1003'],
    'user_id': [501, 502, 503],
    'transaction_time': ['2024-04-15 08:23:11', '2024-04-15 08:45:22', '2024-04-15 09:01:37'],
    'model_score': [0.03, 0.92, 0.95],
    'model_version': ['v2.1.0', 'v2.1.0', 'v2.1.0'],
    'flagged_for_review': [False, True, False]
})

@register_check_function(name="high_score_is_flagged")
def high_score_is_flagged(row):
    return row['model_score'] <= 0.9 or row['flagged_for_review'] is True

model_score_validator = (
    FrameCheck()
    .column('transaction_id', type='string', regex=r'^TXN\d{4,}$')
    .column('user_id', type='int', min=1)
    .column('transaction_time', type='datetime', before='now')
    .column('model_score', type='float', min=0.0, max=1.0)
    .column('model_score', type='float', not_in_set=[0.0], warn_only=True)
    .column('model_version', type='string')
    .column('flagged_for_review', type='bool')
    .custom_check(
        high_score_is_flagged,
        "flagged_for_review must be True when model_score > 0.9"
    )
    .not_null()
    .not_empty()
    .only_defined_columns()
)

result = model_score_validator.validate(df)

if not result.is_valid:
    print(result.summary())

Output:

Validation FAILED
1 error(s), 1 warning(s)
Errors:
  - flagged_for_review must be True when model_score > 0.9 (failed on 1 row(s))
Warnings:
  - Column 'model_score' contains disallowed values: [0.0].

Identify Problem Records

invalid_rows = result.get_invalid_rows(df, include_warnings = True)
transaction_id user_id transaction_time model_score model_version flagged_for_review
TXN1003 503 2024-04-15 09:01:37 0.95 v2.1.0 False

📋 Save & Reuse Validation Rules

model_score_validator.save('transaction_validator.json')

loaded_validator = FrameCheck.load('transaction_validator.json')

result = loaded_validator.validate(df)

📊 Comparison with Other Tools

FrameCheck is designed to be concise and pandas-native. For full comparisons with other packages:


License

MIT


Contact

LinkedIn Badge

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

framecheck-0.5.1.tar.gz (48.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

framecheck-0.5.1-py3-none-any.whl (54.4 kB view details)

Uploaded Python 3

File details

Details for the file framecheck-0.5.1.tar.gz.

File metadata

  • Download URL: framecheck-0.5.1.tar.gz
  • Upload date:
  • Size: 48.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.8.20

File hashes

Hashes for framecheck-0.5.1.tar.gz
Algorithm Hash digest
SHA256 314751a19bdde01f655be98e5acb2da3498669576937b95879c34f8aca9594eb
MD5 0bee267a2cc223c1ecce1879045c809c
BLAKE2b-256 7e619d4d1eb9bc2924cadbf5a530482881b821a114f811c79522146b2bcef1f5

See more details on using hashes here.

File details

Details for the file framecheck-0.5.1-py3-none-any.whl.

File metadata

  • Download URL: framecheck-0.5.1-py3-none-any.whl
  • Upload date:
  • Size: 54.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.8.20

File hashes

Hashes for framecheck-0.5.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d24bd28a0ecbde2d5efa7ea49f91bdc1f7c4f7daa554ee1941cc47f98a925ad0
MD5 9c85bfa83acaa6f2a75165bc4e940067
BLAKE2b-256 0fd94ce5a7fd1b778d9ebe3b7201db159c5200d96830d16cdf92ffaa95e6c2cf

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page