DataCritic

DataCritic is a Python package for dataset quality analysis and evidence-based cleaning recommendations.

Instead of only detecting missing values or outliers, DataCritic compares possible cleaning strategies and shows how those decisions can affect the observed data.

Why DataCritic?

Data cleaning is not just about removing bad values.

A cleaning decision can change:

• sample size
• mean
• median
• variance
• standard deviation
• observed distributions
• potential outlier structure

DataCritic helps make those effects visible before a cleaning strategy is applied.

Features

• Dataset profiling
• Missing-value analysis
• Missing-value strategy comparison
• Outlier detection using the IQR method
• Outlier treatment comparison
• Evidence-based recommendations
• Analysis-aware recommendation modes
• Explainable reasoning
• Impact metrics
• Audit-style recommendation reports
• Edge-case validation

Installation

pip install datacritic

Quick Start

import pandas as pd

from datacritic import DataCritic

df = pd.DataFrame({
    "age": [21, 22, 23, None, 25],
    "salary": [30000, 32000, 31000, 35000, 1000000],
    "city": [
        "Delhi",
        "Mumbai",
        "Delhi",
        None,
        "Mumbai",
    ],
})

critic = DataCritic(df)

Dataset Profile

profile = critic.profile()

print(profile)

The profile includes:

• number of rows
• number of columns
• data types
• missing values
• missing percentages
• duplicate rows
• constant columns

Missingness Analysis

result = critic.missingness()

print(result)

This identifies columns containing missing values and reports their missing-value counts and percentages.

Outlier Analysis

result = critic.outliers()

print(result)

DataCritic currently uses the IQR method for numeric outlier detection.

Outlier Treatment Comparison

For numeric columns, DataCritic can compare:

critic.outlier_candidates("salary")

Available strategies:

keep
remove_rows
clip

Impact can be evaluated with:

critic.outlier_impact(
    "salary",
    "clip",
)

Or all strategies can be compared:

critic.compare_outliers("salary")

Missing-Value Strategies

DataCritic can compare:

drop_rows
drop_column
mean
median
mode

For example:

critic.compare("salary")

The comparison can show changes in:

• retained rows
• missing values
• mean
• median
• standard deviation
• variance

Recommendations

DataCritic provides recommendations based on observed dataset characteristics.

result = critic.recommend(
    "salary",
    task="exploration",
)

print(result)

Supported analysis objectives:

exploration
prediction
reporting

These objectives provide context for the recommendation and its explanation. They do not replace validation within a downstream analysis or machine-learning pipeline.

Explainable Reports

Generate a readable recommendation report:

report = critic.recommendation_report(
    "salary"
)

report.show()

The report includes:

• detected dataset issues
• recommended strategy
• strategy comparisons
• outlier impact
• evidence
• reasoning
• limitations

Example Report

============================================================
                    DATACRITIC REPORT
============================================================

DATASET ISSUE
------------------------------------------------------------
Column          : salary
Task            : exploration
Missing values  : 1
Missing percent : 12.50%
Outliers        : Detected 1 potential outlier(s).

RECOMMENDED STRATEGY
------------------------------------------------------------
MEDIAN

STRATEGY COMPARISON
------------------------------------------------------------
...

OUTLIER IMPACT
------------------------------------------------------------
...

EVIDENCE
------------------------------------------------------------
...

REASONING
------------------------------------------------------------
...

LIMITATIONS
------------------------------------------------------------
...

Design Philosophy

DataCritic follows a simple principle:

Don't just clean the data. Evaluate what you may change by cleaning it.

The package is designed to make cleaning decisions more transparent by exposing measurable changes produced by candidate strategies.

Limitations

Recommendations are based on observed dataset characteristics.

DataCritic does not claim to determine the true missingness mechanism from observed data alone.

Recommendations should be validated against the requirements of the specific dataset, analysis, and downstream model.

For predictive modelling, preprocessing should be evaluated inside the appropriate training and validation pipeline.

Testing

The project uses pytest.

Run the test suite with:

pytest -q

The test suite covers:

• dataset validation
• profiling
• missingness analysis
• strategy generation
• impact analysis
• outlier detection
• outlier treatment
• recommendations
• reporting
• edge cases

Project Structure

DataCritic/
├── examples/
├── src/
│   └── datacritic/
│       ├── __init__.py
│       ├── critic.py
│       ├── impact.py
│       ├── missingness.py
│       ├── outliers.py
│       ├── profiler.py
│       ├── recommendation.py
│       ├── report.py
│       └── strategies.py
├── tests/
├── README.md
├── pyproject.toml
└── .gitignore

License

MIT License.