omni_eda

Fully automated, production-grade Exploratory Data Analysis for any pandas DataFrame.

omni_eda inspects a dataset, profiles every column, finds data-quality problems, computes statistics for every data type, generates dozens of visualizations, detects outliers and correlations, suggests feature engineering steps, optionally evaluates a target column, and renders everything into a polished, shareable HTML report — in one function call.

from omni_eda import OmniEDA

OmniEDA("data.csv").generate_report("report.html")

Contents

• Features
• Installation
• Quick start
• Command-line interface
• Configuration
• What's in the report
• Export formats
• Optional cleaning
• Architecture
• Performance notes
• Development
• License

Features

Installation

pip install omni-eda

or, from a local checkout:

pip install -e .

Some visualizations (mosaic plots, ACF/PACF, correlation networks, UMAP) and file formats (Parquet/Feather) need extra libraries. Install everything with:

pip install "omni-eda[extra]"

Every feature that needs an extra dependency degrades gracefully (it's skipped, with a log message) if that dependency isn't installed — nothing crashes.

Quick start

import pandas as pd
from omni_eda import OmniEDA

df = pd.read_csv("customers.csv")

eda = OmniEDA()
results = eda.run(df)          # run the full pipeline
eda.summary()                  # quick console overview
eda.generate_report("report.html")

One-liner form:

from omni_eda import OmniEDA

OmniEDA("customers.csv").generate_report("report.html")

With a target column (enables class-imbalance checks, feature importance, ANOVA/chi-square tests, and ROC/PR/lift curves for binary targets):

from omni_eda import OmniEDA, EDAConfig

config = EDAConfig(target_column="churned", theme="corporate")
eda = OmniEDA(config=config)
eda.run(df)
eda.export(formats=["html", "excel", "dashboard"])

See examples/basic_usage.py and examples/advanced_usage.py for complete, runnable scripts.

Command-line interface

# Analyze a file and write an HTML report
omni-eda run data.csv -o report_output -f html

# With a target column, several export formats, and a lighter run for a quick look
omni-eda run data.csv --target price -f html json excel --sample-rows 50000

# Run just the (conservative) auto-cleaning pipeline
omni-eda clean data.csv -o cleaned.csv

Run omni-eda run --help for the full list of flags (theme, ignored columns, output formats, quiet mode, etc.).

Configuration

Every tunable knob lives in a single EDAConfig dataclass (see omni_eda/config.py for the full list with defaults):

from omni_eda import EDAConfig

config = EDAConfig(
    title="Customer Churn - EDA",
    target_column="churned",
    ignore_columns=["customer_id"],
    theme="dark",                          # light | dark | minimal | corporate
    high_correlation_threshold=0.85,
    outlier_methods=["iqr", "zscore", "isolation_forest", "lof"],
    sample_for_plots=20_000,               # cap rows used for plotting on huge datasets
    max_rows_for_expensive_ops=50_000,     # cap rows used for correlation/outlier/model fits
    export_formats=["html", "json", "excel"],
)

What's in the report

The generated HTML report includes: a dataset overview (rows, columns, memory, column-type breakdown), the full data-quality issue list (severity-tagged), missing-value analysis with matrix/heatmap/dendrogram/bar visualizations, per-column statistics and distribution plots, correlation heatmaps and high-correlation pairs, an outlier summary table, bivariate and multivariate visualization galleries, time-series analysis (when datetime columns are present), target analysis (when a target column is configured), feature-engineering suggestions, and a plain-language final summary.

Export formats

eda.export(output_dir="out", formats=["html", "excel", "dashboard"])

Optional cleaning

Cleaning never happens implicitly. Call .clean() explicitly and get back a new DataFrame plus a human-readable log of exactly what changed:

cleaned_df = eda.clean(steps=["dedup_rows", "dedup_columns", "convert_dtypes", "infinities"])

Available steps: dedup_rows, dedup_columns, drop_constant, fill_missing, convert_dtypes, strip_whitespace, infinities (the default pipeline runs the non-destructive subset of these; fill_missing and drop_constant are opt-in since they change the data more aggressively). Lower-level building blocks (clip_outliers_iqr, encode_categoricals, scale_numeric_columns, ...) are available directly from omni_eda.cleaning for custom pipelines.

Architecture

omni_eda/
├── __init__.py            # public API: OmniEDA, EDAConfig
├── analyzer.py             # OmniEDA orchestrator - runs the full pipeline
├── config.py                # EDAConfig dataclass (every tunable setting)
├── loaders.py                # CSV/Excel/Parquet/Feather/JSON/SQL/folder loading
├── detection.py               # column type & semantic-role detection
├── statistics.py               # descriptive statistics per dtype
├── cleaning.py                   # optional, auditable cleaning operations
├── quality.py                     # data quality issue detection
├── correlation.py                  # Pearson/Spearman/Kendall/Cramer's V/MI/distance corr
├── outliers.py                      # Z-score/IQR/Isolation Forest/LOF/DBSCAN/Elliptic Env.
├── missing.py                        # missing-value analysis & visualization
├── visualization.py                   # ~40 plot functions + the PlotEngine orchestrator
├── feature_engineering.py              # rule-based feature suggestions
├── target_analysis.py                   # class imbalance, tests, importance, ROC/PR/lift
├── themes.py                             # matplotlib/seaborn/report color themes
├── report.py                              # HTML/Markdown/console report builder (Jinja2)
├── export.py                               # HTML/MD/JSON/Excel/PDF/CSV/figure export
├── dashboard.py                             # self-contained interactive HTML dashboard
├── logger.py                                 # package-wide logging + progress bars
├── utils.py                                   # shared helpers (sampling, caching, ...)
├── cli.py                                      # `omni-eda` command-line interface
└── templates/report.html.j2                     # the HTML report template

Every pipeline stage in OmniEDA.run() is wrapped so a failure in one stage (an exotic dtype breaking a single plot, say) is logged and skipped rather than aborting the whole run.

Performance notes

omni_eda is built to stay usable on large datasets without needing a distributed runtime:

• Expensive operations (correlation, outlier detection, PCA/t-SNE, model fitting) sample down to max_rows_for_expensive_ops (default 50,000 rows) rather than processing everything.
• Plotting samples down to sample_for_plots (default 20,000 rows).
• Duplicate-column detection hashes columns instead of transposing the DataFrame, which is dramatically faster on wide-ish, long DataFrames.
• CSV loading supports chunksize for files too large to read at once.
• Numeric downcasting and category-dtype conversion (omni_eda.utils.optimize_dtypes) are available to cut memory usage before analysis.
• Random Forest / mutual information calls in target analysis use their own bounded samples so a 26-column, 100k-row dataset with a target column finishes in roughly a minute on a single CPU core; tune max_rows_for_expensive_ops, n_jobs, and enable_target_modeling / enable_model_based_outliers down further for a quicker look at very large data.

Development

git clone https://github.com/example/omni_eda.git
cd omni_eda
pip install -e ".[dev,extra]"
pre-commit install

pytest                       # run the test suite
pytest --cov=omni_eda        # with coverage
ruff check omni_eda tests    # lint
black omni_eda tests         # format
mypy omni_eda                # type-check

Contributions are welcome — please open an issue or pull request.

License

MIT — see LICENSE.