decisive

A causal-read toolkit that returns a verdict, not a number — and tells you when not to trust it.

Most causal-inference libraries hand you a coefficient and a p-value and leave the hard part — should I believe this, and what should I do about it? — to you. decisive is built around that hard part. You describe the data you have; it routes to the right quasi-experimental design, checks the identifying assumptions loudly, and returns a trust level and a recommendation — or refuses, and points you back to a design that could answer the question.

The core rule: statistical significance never overrides a design failure. A confounded estimate with p < 0.001 still comes back unreliable.

Why it exists

The whole Python causal ecosystem is sharp estimation engines (statsmodels, linearmodels, EconML, DoWhy, …) that assume you have already chosen a design, satisfied identification, and can read a coefficient table. Three things are missing across all of them, and decisive is built to be the layer that has them:

1. Routing — you describe the data; it picks the method.
2. Loud refusal — a failed identifying assumption blocks the causal claim, it doesn't get a footnote.
3. A verdict — effect, uncertainty, robustness, trust, and a ship / don't-ship recommendation gated on trust and on your cost of being wrong — never on a p-value alone.

Plus a ground-truth teaching layer (datasets/) — parameterizable data-generating processes where you set the true effect — so you can watch a method recover the truth when its assumptions hold and produce confident nonsense when they don't.

Install

pip install decisive             # runtime deps: just numpy + pandas. No scipy.
pip install "decisive[plot]"     # + matplotlib, for result.plot() / decision.plot()

60-second tour: the teaching loop

Generate data with a known effect, then watch the verdict track — or refuse — it.

from decisive.datasets import make_did
from decisive import analyze

# same true effect (+3.0) both times; the second injects a pre-existing trend
clean  = make_did(true_effect=3.0, parallel_trends=True,  seed=11)
broken = make_did(true_effect=3.0, parallel_trends=False, seed=11)

analyze(clean,  treatment="treated", outcome="y", time="period", unit="unit", post="post").trust
# 'strong'      → the effect is trustworthy; ships.

analyze(broken, treatment="treated", outcome="y", time="period", unit="unit", post="post").trust
# 'unreliable'  → the estimate is biased to ~+5.7 with p≈0.000, and decisive REFUSES it.

The broken read looks maximally significant. It is refused because the treated group was already diverging from control before treatment — significance cannot launder a broken design.

The three moments

decisive shows up at the three points where expensive mistakes happen, not just at analysis.

Moment A — before the ship: make a holdout cheap and costed

from decisive.plan import plan_holdout

print(plan_holdout(daily_traffic=20_000, baseline_rate=0.04, mde=0.05).summary())
# Hold out 5% (~79,315 units) and run ~80 day(s): you can detect a lift of 5.0%
# or larger (baseline 4.0%). duration solved from your minimum detectable effect.

And the argument for holding out — the salvage penalty if you don't:

from decisive.plan.design_forecast import forecast_no_holdout

print(forecast_no_holdout(daily_traffic=20_000, baseline_rate=0.04, mde=0.05).summary())
# With a holdout: detect a 5.0% lift with randomized, assumption-free inference ...
# Without one: your best read is observational ... capped at 'moderate' trust ...
# The penalty: a confounder ... could bias the estimate by ~2.5% — about 0.5x the
# lift you're trying to detect, often enough to flip ship/kill.

Moment B — after the ship: read what happened honestly

from decisive import analyze

result = analyze(data, treatment="region_treated", outcome="revenue",
                 time="week", unit="region", post="post", min_effect=0.01)
print(result.summary())

Effect: +3.08 (95% CI: +2.71 to +3.44)  ·  design: difference-in-differences
Trust: STRONG
  ✓ parallel pre-trends hold (p=0.80)
  ✓ 18 control units available
  ✓ 24 clusters — cluster-robust inference is sound
  Robustness: ~17.5 standard errors of systematic bias would be needed to erase this.
  Placebo test: permutation p=0.000.
  HonestDiD (robust): survives a post-period trend violation up to ~7.9× the largest pre-trend.

→ Ship: the effect clears your threshold with room to spare.

When the design can't support the claim, the verdict says so and routes you back to Moment A:

Trust: UNRELIABLE
  ✗ pre-trends diverge before treatment (interaction p=0.00) — difference-in-differences is not identified
→ This design can't answer the question. See plan/holdout.py for the holdout that would.

Moment C — the decision handoff: ship on the cost of being wrong

A verdict is not a decision. decide() gates ship / don't-ship on a dollar cost model — the market white space no incumbent occupies — while still respecting the trust layer.

decision = result.decide(value_per_unit=2.0, scale=50_000, ship_cost=40_000, unit_label="user")
print(decision.summary())

Business decision: SHIP
  Breakeven effect: +0.4  (covers the $40,000 ship cost across 50,000 users).
  Effect +3.08  ·  expected value of shipping: $267,669.
  P(net-positive): 100.0%  ·  expected downside if you ship: $0.

→ Ship: the effect covers its cost with room to spare (trust: STRONG).

The trust gate survives the dollar layer: an unreliable read returns inconclusive no matter how large the expected value. Significance can't launder a broken design, and neither can a big number.

What analyze() supports

You describe the columns; the router picks the design from the data's shape — and refuses to mis-dispatch to a method it can't stand behind.

Every method returns the same ExperimentResult, so the trust/verdict/plot/decision surface is identical regardless of which design ran.

The methods, briefly

• Difference-in-differences — cluster-robust standard errors by unit with a Student-t(G−1) critical value (the Bertrand–Duflo–Mullainathan fix + the small-cluster correction), a wild cluster bootstrap CI when clusters are few, and a permutation/placebo cross-check. The pre-trends test is cluster-robust and power-aware: p ≥ 0.05 is not a pass if the test was too weak to rule out a trend large enough to fabricate the effect.
• Staggered adoption — Callaway–Sant'Anna group-time ATT(g,t) with clean (not-yet-treated) comparisons, cluster-bootstrap inference, and a pre-period-ATT parallel-trends block. Recovers the true ATT under dynamic effects where naive two-way fixed effects is biased (the Goodman-Bacon "forbidden comparison").
• Synthetic control — a donor counterfactual via simplex-constrained weights (projected gradient, no scipy). Inference is placebo-in-space permutation (there is no analytic SE); the identifying block fires when the donor pool can't reconstruct the treated unit's pre-period. Caps its own claim honestly.
• Interrupted time series — segmented regression with Prais-Winsten AR(1) GLS inference (ITS residuals are serially correlated), a blocking pre-trend-curvature check, and placebo-in-time robustness. Capped at moderate trust by construction — a single series has no control group.

The trust model (the differentiator)

trust is derived from the assumption checks, never asserted (result.py):

• Any check with severity="block" that fails → trust = "unreliable", the effect is reported as descriptive, and the recommendation becomes inconclusive_run_a_real_test (routing back to plan/). This holds regardless of the p-value.
• Each failing severity="warn" check erodes strong → moderate.
• A method that declines a causal claim (e.g. clustering) reports descriptive_only.

The recommendation is then gated on trust and your min_effect (or, via decide(), your dollar breakeven) — never on significance alone.

Robustness & sensitivity

Every ExperimentResult carries an interpretable robustness statement, not just a p-value:

• Confounder-to-overturn — how many standard errors of systematic bias it would take to erase the effect.

• Placebo tests — permutation (DiD, SC) or placebo-in-time (ITS) p-values.

• HonestDiD (Rambachan–Roth relative magnitudes) — a breakdown value: how many times larger than the largest pre-period violation the post-period parallel-trends violation could be before the effect's CI includes zero. Folded into the DiD verdict, and available standalone with an event-study + sensitivity-curve plot:

  from decisive import honest_did_sensitivity
  hr = honest_did_sensitivity(data, treatment="treated", outcome="y",
                              time="period", unit="unit", post="post")
  print(hr.summary())   # "survives a violation up to ~7.9× the largest pre-trend"
  hr.plot()
  

  Note: this is a deliberately conservative identified-set + sampling-error bound, not the exact Rambachan–Roth fixed-length CI. Labeled as such everywhere.

Diagnostic plots

Every result renders a verdict-colored diagnostic panel (green = strong/ship, amber = moderate, red = unreliable/route-back), so the reasoning is visible, not asserted:

result.plot()          # DiD: trajectories + placebo null + effect/CI
                       # SC: treated-vs-synthetic + gap-spaghetti + effect
                       # ITS: series + counterfactual + placebo-in-time + effect
                       # staggered: event study + overall ATT
decision.plot()        # effect distribution vs breakeven, loss zone shaded

Ground-truth datasets (for learning and validation)

The analog of seaborn.load_dataset for causal inference — each generator carries true_effect and a knob that breaks its identifying assumption, so you can watch a method succeed and fail on data whose truth you set. Four route to a live verdict today; the other two ship runnable numpy recovery snippets.

from decisive.datasets import (
    make_did,                 # 2×2 difference-in-differences        (routes to a verdict)
    make_staggered,           # staggered adoption                   (routes to a verdict)
    make_synthetic_control,   # one treated unit + a donor pool      (routes to a verdict)
    make_its,                 # single interrupted series            (routes to a verdict)
    make_confounded,          # cross-sectional selection bias       (teaching data + fixture)
    make_rdd,                 # sharp regression discontinuity       (teaching data + fixture)
)

Programmatic use

result.to_dict()   # JSON-safe: effect, ci, trust, recommendation, assumptions, robustness, ...

The gallery

Ten copy-runnable lessons live in notebooks/, each setting a known truth and watching the verdict track or refuse it. Every cell runs in CI (tests/test_gallery.py), so the gallery can't rot. Start with 01 — the teaching loop. Highlights: cluster-robust SEs, honest pre-trends, the DGP catalog, DiD-vs-ITS routing, synthetic control, HonestDiD, staggered adoption (Goodman-Bacon), and the dollar decision.

Scope & honest limitations

This release is observational causal reads + the decision layer. Be aware of the edges:

• Experiments are not built yet. The variance-reduction / sequential-testing half of the vision (CUPED, mSPRT, stratification — Tier-2) is stubbed. decisive today is for reading data you already have, plus planning and deciding around it.
• No cross-sectional estimator. Data with no time dimension is refused with guidance (add a pre/post time dimension to unlock a design); selection-on-observables (matching / IPW) is roadmap.
• make_confounded / make_rdd generate ground-truth data but have no wired estimator yet.
• ITS single-series coverage is ~0.88–0.93 and degrades under strong serial correlation (disclosed via the autocorrelation warning); synthetic control placebo inference is coarse with few donors.

See ROADMAP.md for what's next and docs/v0.1-readiness.md for the release status.

Design principles & firewalls

Two structural firewalls are enforced in CI, not by convention:

1. The causal router cannot import descriptive clustering — clustering is for targeting, and can never feed a verdict.
2. Heterogeneous-treatment-effect analysis hard-depends on multiplicity correction — no subgroup effect is ever returned without it.

See CLAUDE.md for the full design rationale, the verdict rules, and the build order — read it before adding methods.

Develop

git clone https://github.com/Aman12x/decisive && cd decisive
pip install -e ".[dev,plot]"        # dev = pytest, plot = matplotlib
pytest                              # the test suite is Monte-Carlo-gated
python notebooks/build_gallery.py   # re-execute the gallery (needs nbformat/nbconvert/ipykernel)

Every method ships with its ground-truth DGP and a Monte-Carlo test (unbiasedness, coverage, verdict behavior). No estimator lands without proof it recovers the truth and its intervals cover.