honest-eda 0.1.0

EDA that admits when there's no signal — wraps nullbic ΔBIC falsification

honest-eda

EDA that admits when there's no signal.

ydata-profiling shows you 200 correlations. 195 are noise. honest-eda shows you the 5 that survive shuffling.

---

The Problem

pandas-profiling, ydata-profiling, and sweetviz report everything. Every correlation, every association, every distribution shift. The result: 200 "insights" in your report, of which 195 are pure chance — artifacts of finite sample size, collinearity, or subtle target leakage.

You end up fitting models on noise, wasting compute on AutoML pipelines that have nothing real to learn, and shipping features that degrade on new data.

The Solution

honest-eda runs a falsification test on every feature-target pair using nullbic — a symbolic regression library with built-in self-falsification via ΔBIC.

For each relation, the verdict is computed against three null hypotheses:

• vs. constant model
• vs. linear model
• vs. target-shuffled distribution

Only relations that survive all three appear in the report. Everything else is hidden.

---

Install

pip install honest-eda

---

Usage

Python

from honest_eda import profile, check

# Generate HTML report — only real signals shown
result = profile("data.csv", target="y", output="report.html")
print(f"Real signal patterns: {len(result.strong)}")
print(f"Fake correlations rejected: {result.noise_count}")
print(f"Leakage suspects: {len(result.leakage_suspects)}")

# CI mode — exit 1 if no real signal
if not check("data.csv", target="y", min_strong=1):
    raise ValueError("No real signal in this dataset!")

CLI

honest-eda profile data.csv --target=y --output=report.html
honest-eda check data.csv --target=y --min-strong=2   # exit 1 if fails

---

What You Get

HONEST EDA REPORT
─────────────────
Columns scanned: 47
Relations tested: 1081

REAL signal (STRONG): 6
  • age × tenure → churn
  • monthly_charges → churn

WEAK signal: 12
NOISE rejected: 1063 (hidden)

Linear-baseline-only features: 23
Leakage suspects: 2  ⚠

No noise, no false confidence. Only findings that hold up under falsification.

---

Killer Features

Leakage detector — When a feature's z-score vs. shuffled target drops below −15, honest-eda flags it as a probable data leak. Catches target-encoded columns, future-information leaks, and accidental label copies before they corrupt your model evaluation.

Linear vs. symbolic dichotomy — honest-eda tells you explicitly when a linear model would suffice. If symbolic regression finds no improvement over OLS, the feature is labeled "linear-baseline-only". No need to run a neural net to discover this.

CI mode — honest-eda check exits with code 1 if the minimum number of strong signals is not met. Drop it in your CI pipeline to block training runs on datasets with no real predictive content.

Pre-modeling triage — Know whether there is exploitable signal before you launch XGBoost or AutoML. Saves hours of compute and avoids the "model trained fine but generalizes to nothing" postmortem.

---

How It Works

For each numeric feature paired with the target:

1. nullbic.discover fits a symbolic expression and records the BIC improvement over the null model.
2. The verdict is assigned:
   • STRONG — beats constant + linear baseline + all shuffled-target permutations.
   • WEAK — beats constant baseline only.
   • NOISE — fails to beat the constant. Excluded from the report.
3. Leakage is flagged when z_vs_shuffled < −15, indicating the feature carries near-perfect information about the target.

The HTML report contains only STRONG and WEAK relations. NOISE is counted and disclosed in the summary, but not displayed.

---

Comparison

	honest-eda	ydata-profiling	sweetviz
Tests vs shuffled target	✅	❌	❌
Reports only real signal	✅	❌	❌
Symbolic formula extraction	✅	❌	❌
Leakage detection	✅	❌	❌
CI mode (exit code)	✅	❌	❌

---

License

MIT. See LICENSE.

---

Citation / Related

honest-eda is built on top of nullbic, a library for symbolic regression with automatic ΔBIC falsification:

• Repository: https://github.com/glogwa68/nullbic

If you use honest-eda in published work, please also cite nullbic.