FitCheck

Zero-boilerplate ML data validation, model evaluation, and drift detection.

FitCheck is a local-first toolkit for answering three questions quickly: Is this dataset healthy? Is this model behaving? Has production data changed? Every workflow produces a self-contained HTML report that can be opened locally and shared in a pull request, Slack, or an incident review.

Why FitCheck

FitCheck is intentionally opinionated. It is zero-config for the common path, never mutates data silently, makes recommendations understandable to beginners, and keeps advanced diagnostics available without making the first command complicated. It does not send telemetry or require a hosted service.

Installation

The repository is currently the canonical install source. Use this command today:

python -m pip install "git+https://github.com/neoline361-art/fitcheck.git"

If the package is published to PyPI in the future, the shorter form will be:

python -m pip install data-fitcheck

For development:

git clone https://github.com/neoline361-art/fitcheck.git
cd fitcheck
python -m pip install -e ".[dev]"

The one-command workflow

fitcheck full data.csv \
  --target label \
  --model model.joblib \
  --reference train.csv \
  --auto-fix \
  --output-dir fitcheck_reports

This creates a dataset report, model evaluation report, optional drift report, and—when issues are found—a transparent fix script. The workflow uses the dataset columns other than label as model features, so it is best suited to a model trained on the same feature schema.

Python API

import fitcheck

issues = fitcheck.check("data.csv", target="label")
metrics = fitcheck.report(model, X_test, y_test)
results = fitcheck.detect_drift("train.csv", "production.csv", method="auto")

The dataset check accepts threshold overrides without requiring a configuration file:

fitcheck.check(
    "data.csv",
    target="label",
    config={
        "missing_warning": 0.05,
        "missing_critical": 0.20,
        "outlier_threshold": 0.01,
    },
)

What FitCheck checks

For drift, method="auto" uses KS on smaller numeric samples and PSI on larger numeric samples. Use method="wasserstein" when a normalized distribution-distance signal is more useful than a hypothesis test.

CLI commands

fitcheck check data.csv --target label
fitcheck check data.csv --missing-warning 0.10 --missing-critical 0.30
fitcheck report model.joblib X_test.npy y_test.npy
fitcheck drift train.csv production.csv --method psi
fitcheck demo

Reports and privacy

FitCheck does not upload input data. HTML reports embed generated plots as base64 data and include only the information derived from the supplied datasets. Model loading uses Python pickle for user-owned artifacts; never load a model file from an untrusted source.

Development and verification

pip install -e ".[dev]"
ruff check fitcheck tests
mypy fitcheck
bandit -r fitcheck/ -x tests
pytest --cov=fitcheck --cov-report=term-missing

The current repository suite contains 44 passing tests and reports approximately 95% total coverage on the supported Python environment.

Large CSVs and contact data

Phone numbers should be stored as strings, not numeric values, so leading zeros and country prefixes are preserved. Names are treated as text values. A quick local check is:

fitcheck check contacts.csv --output contacts_report.html
# Fast, explicit sample review for a very large CSV:
fitcheck check contacts.csv --sample-rows 100000 --output contacts_sample_report.html

FitCheck is designed for in-memory pandas workflows. A file with 1 million rows and a few narrow columns is a reasonable local smoke-test target, but a 10-million-row file may require several gigabytes of RAM depending on string length and pandas version. For a fast schema/sample review, use pandas to create a representative sample before calling FitCheck; do not claim a sample report is a full-dataset audit. FitCheck does not print or transmit raw phone numbers in the terminal, but generated reports can contain previews, so protect report files as sensitive data.

Documentation

License

FitCheck is released under the Apache 2.0 License. See LICENSE.