FlashEDA

Current version: 0.3.0 · Changelog

Analyse any dataset in a few seconds, then get validated cleaning code that runs on the full data, in the engine that fits its size: pandas, polars, DuckDB or PySpark.

FlashEDA analyses a fixed-size random sample, so analysis time barely changes between 100 thousand and 100 million rows.

Install

pip install flasheda            # core
pip install "flasheda[fast]"    # + polars, DuckDB, pyarrow (recommended for big files)
pip install "flasheda[spark]"   # + PySpark

Quick start

import flasheda as fe

# 1. Analyse: any size, a few seconds
report = fe.analyze("orders.csv", target="churn", save_pdf=False)
report.show()                      # console report
report.save_html("report.html")    # or report.save_pdf("report.pdf")

# 2. Clean: get a script for the full data, already checked on the sample
code = fe.clean("orders.csv", target="churn", engine="polars")
code.save("clean_orders.py")       # then: python clean_orders.py

What it does

Cleaning steps

Each step in the generated code has a comment saying why it was added:

• drop likely-ID, mostly-empty, constant and leaky columns
• drop rows where the target is missing, and drop duplicate rows
• convert numbers stored as text ("12.5", "N/A") to numbers
• merge spelling variants ("Delhi", " delhi", "DELHI")
• group rare categories into other
• add <col>_was_missing flags when missingness predicts the target
• fill nulls (median / mode), log-transform skewed columns, clip outliers

Engine choice

Override with fe.analyze(..., engine="duckdb") or report.codegen("duckdb").

Performance

Measured on synthetic messy data:

Examples

• examples/phase1_demo.py: engine choice and code generation at 0.1M / 1M / 5M rows
• examples/phase2_demo.py: target analysis and cleaning on a 1M-row churn CSV

Development

pip install -e ".[dev]"
pytest

License

MIT