sablier-flow 1.0.18

Synthetic alternative-history generation for backtest overfitting detection

sablier-flow

Stop shipping overfit backtests.
Run your strategy on N alternative versions of history that share your data's statistical fingerprint.

---

What it is

sablier-flow is a Python SDK that learns the joint dynamics of your market data — cross-asset correlations, vol clustering, regime structure — and generates synthetic alternative histories that share those statistics but produce different specific paths.

Run your existing backtest on N alternative versions of the same data and turn a single P&L number into a distribution. If your strategy only works on the one history that happened, that's overfit — now measurable.

Install

pip install sablier-flow

Sign up at sablier.ai for an API key (free starter credits cover the entire getting-started notebook).

Quickstart

import sablier_flow as sf
import numpy as np

# 1. Auth (one-time device flow; sets ~/.sablier/credentials)
sf.login()

# 2. Your backtest. Takes a price DataFrame, returns dict[str, float].
def my_backtest(prices):
    rets = prices['SPY'].pct_change().dropna()
    return {'sharpe': float(rets.mean() / rets.std() * np.sqrt(252))} if rets.std() > 0 else {'sharpe': 0.0}

# 3. Load data — bundled demo or your own DataFrame.
df = sf.demo_data()                          # SPY/QQQ/IWM/TLT + 3 macro features, 2010-2023
backtest_window = df.iloc[-252:]             # the slice you'll evaluate

# 4. Train + generate synthetic alternative versions of the backtest window.
fit   = sf.fit(df, features=list(df.columns), data_types=df.attrs['data_types'], horizon=252)
paths = sf.generate(fit.model_id, n_paths=200, like=backtest_window)

# 5. Run your backtest on each synth path and score robustness.
real_result   = my_backtest(backtest_window)
synth_results = [my_backtest(p) for p in paths.as_dataframes()]
report = sf.robustness(real_result, synth_results, primary_metric='sharpe')

print(report.summary())

Examples

Live empirical demos with executed outputs baked in — open them on GitHub and see the numbers immediately:

Notebook	What it proves
📓 Backtest Robustness	At the 0.7 overfit_score threshold: flags 29 of 30 selection-biased lucky strategies (top 30 of a 500-strategy pure-noise pool; lucky family min = 0.670, max = 0.875) vs 0 of 12 false positives on a designed honest family (max = 0.690)
📓 TSTR Predictive Rank	Spearman ρ = +0.7687, 95% bootstrap CI [+0.47, +0.95], p = 1.1e-05, n = 24 — synth ranks predict real OOS ranks
📓 Memorization Audit	NN-distance ratio R = 0.9312 vs replay-floor R = 0.0161 — 57.8× separation, synth is genuinely new
📓 Getting Started	End-to-end SDK tour: login → fit → validate → generate → robustness

Why use it

• One call trains a model that handles cross-asset dependence, regime structure, and tail behavior — no hand-rolled copulas, no block-length tuning
• Per-strategy overfit detection that classical CSCV-PBO can't surface (selection bias from parameter searches)
• Train on synth, deploy on real — sf.predictive_rank_score proves the ranking carries forward, so you don't have to burn real OOS data on strategy selection
• Engine-agnostic: works with pandas, backtrader, vectorbt; LEAN CSV export adapter included

Docs

• docs.sablier.ai — full SDK reference, recipes, concepts
• Quickstart — pip install to first verdict in 5 minutes
• Concepts: why in-sample training is correct
• Concepts: data sourcing + engine integration

License

Apache 2.0 (code) · CC BY 4.0 (docs)