mldebug 0.1.0

A lightweight Python package for comparing datasets and detecting unexpected changes in machine learning systems.

mldebug

A lightweight Python package for comparing datasets and detecting unexpected changes in machine learning systems.

Why mldebug

Machine learning systems fail silently when data changes.

Common production issues include:

• feature distribution drift
• increasing missing values
• unseen categorical values
• training vs production mismatch

mldebug provides a unified way to detect these issues before they become model failures.

What it does

mldebug compares:

• a reference dataset (e.g. training data)
• a current dataset (e.g. production data)

It runs a suite of checks and returns a structured report of detected issues.

Installation

pip install mldebug

Quick Start

from mldebug import run_checks
import numpy as np

reference = {
    "age": np.array([20, 21, 22]),
    "income": np.array([1000, 1200, 1100]),
    "country": np.array(["ES", "ES", "FR"]),
}

current = {
    "age": np.array([30, 35, 40]),
    "income": np.array([900, 800, 850]),
    "country": np.array(["ES", "DE", "DE"]),
}

schema = {
    "age": "numeric",
    "income": "numeric",
    "country": "categorical",
}

report = run_checks(reference=reference, current=current, schema=schema)

Inspect detected issues

Human-readable output

for issue in report.issues:
    print(issue)

[WARNING] psi_drift - country: PSI drift detected (18.0152)

Summary

print(report.summary())

{
  "total": 1,
  "by_severity": {
    "info": 0,
    "warning": 1,
    "critical": 0
  },
  "status": "issues_detected"
}

Structured output

print(report.to_dict())

{
  "issues": [
    {
      "name": "psi_drift",
      "metric": "psi",
      "severity": "warning",
      "message": "country: PSI drift detected (18.0152)",
      "feature": "country",
      "value": 18.01521528247136,
      "threshold": 0.2
    }
  ]
}

Logs

for line in report.to_logs():
    print(line)

[WARNING] psi_drift - country: PSI drift detected (18.0152)

Supported Checks

mldebug runs a combination of:

Numeric features

• Kolmogorov–Smirnov test (KS test)
• missing value rate changes

Categorical features

• Population Stability Index (PSI)
• category distribution changes

Documentation

See documentation pages.

Status

Active development (v0.x). APIs may evolve before v1.0.0.

See CHANGELOG.md for version history and updates.

Development Setup

Requirements

• Ubuntu 24.04.4 (recommended) or WSL
• git
• direnv

Environment Setup

git clone https://github.com/anpenta/mldebug
cd mldebug
direnv allow

Development Workflow

Tasks are managed via poe (available in the project environment via direnv).

Run tests

poe test

Run linting

poe lint

Check linting

poe lint-check

Run full CI parity checks

poe test-all

poe lint-check-all

CI/CD

CI runs multi-Python version testing and linting. All pull requests must pass the checks before merging.

See CI workflow for details.

Contributing

We welcome contributions.

1. Clone the repository
2. Create a feature branch
3. Make your changes
4. Ensure all CI checks pass
5. Open a pull request

Dependency Management

Dependencies are managed using uv and defined in pyproject.toml.

License

See LICENSE.