Skip to main content

oss-backtest-kit

CI PyPI Python versions License: MIT Open in Colab

Is your backtest real, or did you just try enough ideas until one looked good? A small, honest toolkit for crypto futures research: verified public data, a no-look-ahead backtest engine, and the statistics that tell you how much of your Sharpe ratio is luck.

Equity curve of the best of 9 variants with PSR and DSR computed by the library; synthetic data, not a trading result

Synthetic data, not a trading result. The best of 9 moving-average variants has an average return just above zero (Sharpe ratio slightly positive, PSR 0.53), but the equity curve still ends a little below its start (0.97), because volatility drags down compounded growth; after charging for the 9 tries the DSR is 0.07, so there is no evidence of skill. Reproduce it with examples/equity_psr_dsr.py.

Highlights

  • Overfitting checks built in. Deflated Sharpe Ratio, Probabilistic Sharpe Ratio and Probability of Backtest Overfitting (CSCV) adjust your result for how many variants you tried, so a lucky curve is easier to spot before it costs money.
  • Data you can trust. Every Binance public archive file is checked against its published SHA-256 before it is saved; a manifest records what you have; gaps and duplicate bars are reported, not silently filled.
  • No look-ahead by design. Your strategy is called at a bar's close and sees only bars up to that point; orders fill at the next bar's open with slippage and fees.
  • Costs that bite. Perpetual-futures funding is included, and a stressed cost level lets you check whether an edge survives worse fills.
  • Small and readable. Pure Python with only numpy and scipy; 62 offline tests; no account, API key or paid data needed.
  • MIT licensed code (the market data has its own terms; see below).

Who it is for

Individual researchers and students who write their own strategies in Python and want a quick, sober second opinion on a backtest. It is not a trading bot, does not place orders and does not connect to any exchange account.

What is inside

A small research toolkit for crypto strategies, in three parts:

  1. Multiple-testing statistics (oss_backtest_kit.stats): Sharpe, Probabilistic Sharpe Ratio (PSR), Deflated Sharpe Ratio (DSR), Probability of Backtest Overfitting (CSCV), effective number of trials, circular block bootstrap, Wilson intervals.
  2. Public data download and verification (oss_backtest_kit.data): fetches Binance public archive files, verifies each against its published SHA-256 checksum, records a manifest, and checks bar continuity. A REST helper exports the most recent days as separate CSV files (not loaded by the engine).
  3. A simple, config-driven backtest engine (oss_backtest_kit.engine) for a portfolio of perpetual futures, with funding and a two-level cost model.

DISCLAIMER

This project is for research and education only. It is not investment advice. It comes with no warranty of any kind; you use it at your own risk. Backtest results are not live results: past or simulated performance says little about future outcomes, and you can lose money trading.

Status

Early release (version 0.1.0). The project is maintained: bug reports and pull requests are welcome. The API may still change before 1.0; see CHANGELOG.md.

Try it in 30 seconds

No data download and no account needed:

pip install oss-backtest-kit
python -c "import numpy as np; from oss_backtest_kit import stats; r = np.random.default_rng(0).normal(0.0005, 0.01, 500); print('PSR', stats.psr(r)); print('DSR', stats.dsr(r, n_trials=20, trial_sharpes=[0.01, 0.03, 0.02])[0])"

The second number is lower than the first: the Deflated Sharpe Ratio charges you for the 20 variants you tried.

Requirements and install

  • Python 3.9 or newer, with numpy and scipy (installed automatically below).
  • Internet access only if you download data; everything else, tests included, runs offline.
  • Some Python knowledge: you write your strategy as a small Python function.
git clone https://github.com/ww7411/oss-backtest-kit.git
cd oss-backtest-kit
pip install -e .

Tests use pytest (install the test extra first: pip install -e ".[test]"). The stats, engine and data test files can also be run directly with python3.

Quick start

Download public data (nothing is downloaded unless you run this; the output folder is yours to choose):

python -m oss_backtest_kit.data archive --kinds klines,fundingRate --symbols BTCUSDT \
    --interval 1d --from 2024-01 --to 2024-06 --out ./market_data
python -m oss_backtest_kit.data check --out ./market_data \
    --kind klines --symbol BTCUSDT --interval 1d

Kinds: klines (needs --interval), markPriceKlines, indexPriceKlines, premiumIndexKlines, fundingRate, metrics. Files that fail the checksum are neither saved nor recorded; existing files are never overwritten. Recent bars not yet in the archive can be exported with the rest subcommand; it writes standalone CSV files under <out>/rest/ for inspection or your own merging. They are not added to the manifest and load_market_data does not read them, so a backtest ends at the last archive file.

Run the moving-average crossover example (synthetic data, nothing downloaded):

python3 examples/ma_crossover.py

Draw the effect chart above (needs matplotlib, an optional extra that is not a runtime dependency):

pip install -e ".[examples]"
python3 examples/equity_psr_dsr.py

Or open the quick start notebook in Colab (synthetic data, no download).

Use the statistics:

import numpy as np
from oss_backtest_kit import stats

returns = np.random.default_rng(0).normal(0.0005, 0.01, 500)
print(stats.psr(returns))                                # P(true Sharpe > 0)
value, benchmark, var = stats.dsr(returns, n_trials=20,  # penalise 20 tries
                                  trial_sharpes=[0.01, 0.03, 0.02])
print(stats.wilson(7, 50))                               # 95% interval for 7/50

Run a backtest on your own downloaded data:

from oss_backtest_kit.engine import EngineConfig, load_market_data, run_backtest

data = load_market_data("./market_data", ["BTCUSDT"], interval="1d")
cfg = EngineConfig(symbols=["BTCUSDT"], max_leverage=1.0, rebalance_threshold=0.02)

def strategy(t, history):          # called at the close of bar t, sees bars 0..t only
    return {"BTCUSDT": 1.0}        # sum(|weights|) must not exceed max_leverage

result = run_backtest(data, strategy, cfg, cost_level="stressed")
print(result.summary())            # result.returns feeds stats.psr / stats.dsr

Modules

Module Purpose
stats PSR, DSR, PBO (CSCV), effective trials, block bootstrap, Wilson interval
data.archive Archive download with checksum verification, manifest, exclusive-create writes
data.rest_tail Recent closed bars and funding from public REST endpoints
data.check Manifest SHA check, gap and duplicate detection
engine Config, market data loader, backtest loop, result summary

The data client allows a single host per task, uses https only, ignores environment proxy settings, refuses redirects, bounds response size and time, and stops when free disk space is low.

Tests

pip install -e ".[test]"
python3 -m pytest

All tests run offline with synthetic data and fake network responses.

Known limitations

  • Fills are modelled from OHLC bars (decision at a bar close, fill at the next open with slippage and a proportional fee). Real fills depend on order books, latency and size.
  • Funding is modelled as position x price x rate at each funding time, using the open price of the bar that contains the event; margin, liquidation and exchange rules are not modelled beyond stopping at zero equity (limited liability: positions are closed at that bar's close and any loss beyond zero is reported as ruin_shortfall, so sum(pnl_by_symbol) + ruin_shortfall equals the change in equity).
  • Survivorship bias: the symbols you choose are the ones that exist today; delisted symbols are not included unless you add them.
  • Cost defaults (base and stressed) are placeholders, not estimates for any venue. Set your own numbers.
  • Statistics assume the return series is a fair sample; PSR/DSR/PBO reduce, but do not remove, the risk of overfitting.

Data terms

This kit ships code only and never any data. It downloads from two different sources, which have different terms. Nothing here is legal advice; read the current terms yourself before downloading, using or sharing any data.

1. Archive files from data.binance.vision (archive subcommand). These are described in the binance/binance-public-data repository (https://github.com/binance/binance-public-data), whose TERMS_AND_CONDITIONS.md states a CC BY-NC-SA 4.0 licence (https://creativecommons.org/licenses/by-nc-sa/4.0/). As we read that licence, it means:

  • Attribution: if you share the data or anything derived from it (cleaned files, aggregated tables, published charts of the data), credit the source, link the licence, and say whether you changed it.
  • NonCommercial: no commercial use of the data.
  • ShareAlike: data you derive from it and share must be shared under the same licence.

2. Recent data from the REST API at fapi.binance.com (rest subcommand). This is not covered by the archive licence above. It falls under the exchange's own API and website terms of use, which this project has not reviewed; we make no statement about what they allow. Check them before using or sharing REST data.

You alone are responsible for complying with the terms of each source and with the exchange's rules.

License

MIT; see LICENSE. The licence covers this code only, not any market data you download with it (see the data terms above).

Metadata

Release files for oss-backtest-kit 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for oss-backtest-kit 0.1.0
File Size Uploaded Provenance Actions
oss_backtest_kit-0.1.0.tar.gz 166.4 kB With provenance Details for oss_backtest_kit-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for oss-backtest-kit 0.1.0
File Interpreter ABI Platform Actions
oss_backtest_kit-0.1.0-py3-none-any.whl Python 3 none any Details for oss_backtest_kit-0.1.0-py3-none-any.whl

Total release size: 195.8 kB

Release files / oss_backtest_kit-0.1.0.tar.gz

Download URL oss_backtest_kit-0.1.0.tar.gz
Size 166.4 kB
Tags Source
SHA-256 checksum
How to use checksums
3fb3cef26c4a342e27b146e0d4c7d067450522d2d217cd6778945cd74c9bd073
BLAKE2b-256 checksum
How to use checksums
74a1e493c789b285300eb5ee4c1351e7e1fb0cc27375818f4d169d22200db69a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

Release files / oss_backtest_kit-0.1.0-py3-none-any.whl

Download URL oss_backtest_kit-0.1.0-py3-none-any.whl
Size 29.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b4fa5b82c1cbd77bd091ec7ba2e9f5dc2a65567bef9482a7112f7ef9c1ab53c1
BLAKE2b-256 checksum
How to use checksums
54f65686621e9223a0709b73b2fe62e206f369a2faf35bf07f0ff8d6ae0e31e3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

Security for oss-backtest-kit 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for oss_backtest_kit-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page