Skip to main content

Lightweight, readable, and extensible validation for pandas DataFrames.

Project description

Project Logo

CI codecov Documentation Status PyPI Python

Lightweight, flexible, and intuitive validation for pandas DataFrames.
Define expectations for your data, validate them cleanly, and surface friendly errors or warnings — no configuration files, ever.

🔗 View full documentation →


📦 Installation

pip install framecheck

Try FrameCheck

Open In Colab

Try FrameCheck without installing anything - click the badge above to run an interactive demo notebook in Google Colab.


Main Features

  • Designed for pandas users
  • Simple, fluent API
  • Supports error or warning-level assertions
  • Validates both column-level and DataFrame-level rules
  • No config files, decorators, or boilerplate

Why Should You Validate Your Data Explicitly?

Because during a routine update, a typo like this:

SELECT u.credit_score AS age
FROM users u
JOIN profiles p ON u.id = p.user_id

...fails silently.

Now your age column has values in the 700s.

Your SQL doesn't error, your model still runs — and makes terrible predictions.


🔥 Example: Catch Bad Model Output Before It Hits Production

import pandas as pd
from framecheck import FrameCheck, register_check_function

df = pd.DataFrame({
    'transaction_id': ['TXN1001', 'TXN1002', 'TXN1003'],
    'user_id': [501, 502, 503],
    'transaction_time': ['2024-04-15 08:23:11', '2024-04-15 08:45:22', '2024-04-15 09:01:37'],
    'model_score': [0.03, 0.92, 0.95],
    'model_version': ['v2.1.0', 'v2.1.0', 'v2.1.0'],
    'flagged_for_review': [False, True, False]
})

@register_check_function(name="high_score_is_flagged")
def high_score_is_flagged(row):
    return row['model_score'] <= 0.9 or row['flagged_for_review'] is True

model_score_validator = (
    FrameCheck()
    .column('transaction_id', type='string', regex=r'^TXN\d{4,}$')
    .column('user_id', type='int', min=1)
    .column('transaction_time', type='datetime', before='now')
    .column('model_score', type='float', min=0.0, max=1.0)
    .column('model_score', type='float', not_in_set=[0.0], warn_only=True)
    .column('model_version', type='string')
    .column('flagged_for_review', type='bool')
    .custom_check(
        high_score_is_flagged,
        "flagged_for_review must be True when model_score > 0.9"
    )
    .not_null()
    .not_empty()
    .only_defined_columns()
)

result = model_score_validator.validate(df)

if not result.is_valid:
    print(result.summary())

Output:

Validation FAILED
1 error(s), 1 warning(s)
Errors:
  - flagged_for_review must be True when model_score > 0.9 (failed on 1 row(s))
Warnings:
  - Column 'model_score' contains disallowed values: [0.0].

Identify Problem Records

invalid_rows = result.get_invalid_rows(df, include_warnings = True)
transaction_id user_id transaction_time model_score model_version flagged_for_review
TXN1003 503 2024-04-15 09:01:37 0.95 v2.1.0 False

📋 Save & Reuse Validation Rules

model_score_validator.save('transaction_validator.json')

loaded_validator = FrameCheck.load('transaction_validator.json')

result = loaded_validator.validate(df)

📊 Comparison with Other Tools

FrameCheck is designed to be concise and pandas-native. For full comparisons with other packages:


License

MIT


Contact

LinkedIn Badge

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

framecheck-0.4.4.tar.gz (39.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

framecheck-0.4.4-py3-none-any.whl (45.1 kB view details)

Uploaded Python 3

File details

Details for the file framecheck-0.4.4.tar.gz.

File metadata

  • Download URL: framecheck-0.4.4.tar.gz
  • Upload date:
  • Size: 39.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.8.20

File hashes

Hashes for framecheck-0.4.4.tar.gz
Algorithm Hash digest
SHA256 52541c8c9c65bffb168a97374cc56445944213b13a32623707f330aeaafdf1d6
MD5 e2d4054b48e3b24da8f860bbc8fcb98a
BLAKE2b-256 005f0a1527ff58d7625e6c5091c92cb624d8c4d82e1d69e0ce5e4d1a51a989dc

See more details on using hashes here.

File details

Details for the file framecheck-0.4.4-py3-none-any.whl.

File metadata

  • Download URL: framecheck-0.4.4-py3-none-any.whl
  • Upload date:
  • Size: 45.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.8.20

File hashes

Hashes for framecheck-0.4.4-py3-none-any.whl
Algorithm Hash digest
SHA256 7ffd8766319a87881f71b6d9664ff5c14c7da612ee045d9ffe2f51f2b0e93ef8
MD5 dfb2fdff46acb7a5d0a6e42f182fd243
BLAKE2b-256 9aaeaae0223d197d7f14e2ecde36f61fbd586042d0359dd5e1a7168db0247b85

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page