backtest-audit 1.0.0

Static analysis tool for detecting look-ahead bias, data leakage, and statistical errors in Python backtesting code.

backtest-audit

Static analysis for Python backtesting code.
Catches look-ahead bias, data leakage, and statistical errors — before they invalidate your results.

pip install backtest-audit
backtest-audit check ./strategies/

---

Why

A backtest that looks profitable is often built on data the strategy could never have had. Look-ahead bias, data leakage, and missing transaction costs are the three most common ways a backtest lies to you. backtest-audit catches these at the code level, without running your strategy.

---

Output

── strategies/mean_reversion.py ─────────────────────── 2 issues ──

  [LAB001] Negative shift — future data pulled into current timestep  line 34
  df['signal'] = df['close'].shift(-5)
  ↳ Replace shift(-n) with shift(n). Negative periods reference future rows, invalidating the backtest.

  [LEAK002] fit_transform() detected — ensure this is called only on training data  line 61
  X_scaled = scaler.fit_transform(X)
  ↳ Replace fit_transform(X) with fit(X_train).transform(X_train), then transform(X_test) separately.

──────────────────────────────────────────────────────────────────────
1 error, 1 warning

---

Rules

Code	Category	Description
LAB001	Look-Ahead Bias	shift(-n) pulls future prices into current signal
LAB002	Look-Ahead Bias	pct_change(-n) computes forward returns
LEAK001	Data Leakage	fit() called with no train_test_split detected
LEAK002	Data Leakage	fit_transform() may be called on unsplit data
STAT001	Statistical Malpractice	Sharpe ratio assigned without annualization factor
STAT002	Statistical Malpractice	No transaction cost or slippage variable detected

Full documentation with explanations and code examples: docs/rules/

---

Usage

# Check a single file
backtest-audit check my_backtest.py

# Check a directory
backtest-audit check ./strategies/

# Output as JSON (for CI pipelines)
backtest-audit check ./strategies/ --format json

# Ignore specific rules
backtest-audit check ./strategies/ --ignore LAB002,STAT002

# List all rules
backtest-audit rules

---

Installation

pip install backtest-audit

Requires Python 3.8+. The only dependency is click.

---

How It Works

backtest-audit parses your Python source files into an abstract syntax tree (AST) and walks the tree looking for known problematic patterns. It does not execute your code. Each rule is a pattern matcher — it looks for a specific construct that is known to produce invalid backtest results.

This means it is fast, safe to run on any codebase, and produces no false positives from runtime behavior.

---

Limitations

• Does not run your backtest or evaluate strategy profitability
• Cannot detect all forms of look-ahead bias — only statically visible patterns
• LEAK001 checks for train_test_split presence in the file, not correct application
• Manual chronological splits (e.g., df[:split_date]) are not yet recognized
• Does not support non-Python backtesting frameworks

---

License

MIT