wranglefy

Automated data import and wrangling from many formats.

wranglefy is a small, pandas-powered library that takes the friction out of the first mile of data work: getting messy files of any common format into a clean DataFrame. Point it at a file and it figures out the format; hand it a frame and it tidies column names, normalises missing values, infers types, de-duplicates rows, and more — in one call or as a readable, chainable pipeline.

import wranglefy as of

df = of.read("sales.xlsx")            # format auto-detected from extension/content
clean = of.auto_wrangle(df)           # snake_case cols, typed, de-duped, tidy NaNs

# …or do both at once:
clean = of.load("sales.csv.gz", wrangle=True)

Why

Real datasets arrive as CSVs with "N/A" sprinkled through them, Excel exports with "First Name" columns, gzipped JSON lines, the occasional Parquet file, and worse. wranglefy collapses the repetitive boilerplate of detecting formats and cleaning frames into a couple of well-tested calls, while staying out of your way when you need full control (every reader passes **kwargs straight through to pandas).

Installation

pip install wranglefy

The core install only depends on pandas/numpy and reads CSV, TSV, JSON, JSON Lines, and Pickle. Heavier formats live behind optional extras:

pip install "wranglefy[excel]"      # .xlsx / .xls / .ods
pip install "wranglefy[parquet]"    # Parquet / Feather / ORC (pyarrow)
pip install "wranglefy[html]"       # HTML tables
pip install "wranglefy[xml]"        # XML
pip install "wranglefy[spss]"       # SPSS .sav
pip install "wranglefy[all]"        # everything

Supported formats

Reading data

import wranglefy as of

# Single file — format detected from extension, then content if needed.
df = of.read("data.parquet")

# Force a format (e.g. an extensionless file or a buffer).
df = of.read(file_obj, format="csv")

# Read & concatenate many files; tag each row with its source.
df = of.read_many("exports/2024-*.csv", add_source_column=True)

# Inspect what wranglefy can do.
of.supported_formats()
of.detect_format("archive/data.json.gz")   # -> "json"

Wrangling data

One-shot

clean = of.auto_wrangle(df)

The default pipeline: clean column names → strip strings → standardise missing values → drop empty rows/columns → infer types → de-duplicate. Every step is a keyword you can switch off, and fill_missing / drop_constant can be switched on:

clean = of.auto_wrangle(
    df,
    infer_types=True,
    dedupe=False,
    drop_constant=True,
    fill_missing="median",   # or True for the "auto" strategy
    optimize_memory=True,    # downcast floats/ints & categorise strings
    redact_pii=True,         # auto-hash emails, ssns, etc.
    clip_outliers=True,      # clip numeric outliers using IQR
)

Chainable pipeline

For full control, use the Wrangler directly. It copies the input (never mutates it) and records each operation in .log:

from wranglefy import Wrangler

wr = (
    Wrangler(df)
    .clean_columns()                      # "First Name" -> "first_name"
    .strip_strings()
    .standardize_missing(["TBD", "???"])  # extend the default sentinel set
    .drop_empty(axis="both")
    .infer_types(threshold=0.95, category=True)
    .fill_missing("auto")
    .clip_outliers(method="iqr")          # cap statistical anomalies
    .redact_pii(method="mask")            # *** out sensitive columns
    .optimize_memory()                    # shrink memory footprint safely
    .dedupe(subset=["id"])
)

clean = wr.to_frame()
print(wr.log)   # ['clean_columns', 'strip_strings', ...]

# Drop into raw pandas whenever you need to:
wr.apply(lambda d: d[d["score"] > 0])

Operations at a glance

Development

pip install -e ".[dev]"
pytest

License

MIT © Devisri Bandaru