dataclean-kuntal 0.1.2

Auto-detect and fix messy data in one function call

dataclean

Auto-detect and fix messy data in one function call.

Install

pip install dataclean-kuntal

Quick start

import dataclean as dc

clean_df, report = dc.clean("messy_data.csv")
report.summary()

What it fixes

Problem	Example	Fix
Date format chaos	2024-01-15, 15/01/2024, Jan 15 2024	Normalized to YYYY-MM-DD
Hidden nulls	"N/A", "null", "?", "-"	Unified to real NaN
Mixed types	25, "thirty", 25.0	Coerced to dominant type
Sneaky duplicates	John Smith vs john smith	Fuzzy deduplication
Encoding errors	Jos\xef	Repaired to José
Inconsistent labels	"USA", "US", "U.S.A"	Merged via fuzzy matching
Column name mess	"First Name ", "AGE(Years)"	Normalized to snake_case
Outliers	9999999 in salary column	Flagged with context

API

import dataclean as dc

# clean any file or DataFrame
clean_df, report = dc.clean("data.csv")
clean_df, report = dc.clean("data.xlsx")
clean_df, report = dc.clean("data.json")
clean_df, report = dc.clean(df)  # existing DataFrame

# use individual fixers
df = dc.fix_columns(df)
df = dc.fix_nulls(df)
df = dc.fix_dates(df)
df = dc.fix_duplicates(df, method="fuzzy")
df = dc.fix_categories(df)
df = dc.flag_outliers(df)

# audit report
report.summary()       # print human-readable summary
report.to_df()         # pandas DataFrame of all changes
report.to_dict()       # list of dicts
report.changes_for("nulls")  # filter by fixer

Options

clean_df, report = dc.clean(
    source="data.csv",
    fix="all",                  # or list: ["columns", "nulls", "dates"]
    output="clean.csv",         # optional save path
    fuzzy_duplicates=True,      # fuzzy deduplication
    outlier_method="iqr",       # "iqr" or "zscore"
    category_threshold=85,      # fuzzy match threshold
    duplicate_threshold=85,     # fuzzy dedup threshold
)

Author

KUNTALinGITHUB — Kuntal Pal, Independent Researcher, Kolkata, India

License

MIT