sharpebench (Python)
Is my Sharpe real, or an artifact of luck and multiple testing?
Python distribution of SharpeBench's luck-robust quantitative evaluation
statistics, a pyo3 binding over the same deterministic Rust kernel used by the
SharpeBench CLI and the @general-liquidity/sharpebench npm package. Bring your own return series;
everything takes plain numeric sequences (lists, tuples, numpy arrays,
df["ret"].to_numpy()) and returns plain floats, lists and dicts.
import numpy as np
from sharpebench import is_my_sharpe_real, bootstrap_dsr_ci
returns = df["strategy_ret"].to_numpy() # per-period, NOT annualized
# n_trials is the honest one: how many variants did you try before keeping this?
# periods_per_year says what a row is (default 252, daily bars).
v = is_my_sharpe_real(returns, n_trials=200, periods_per_year=252)
print(v["sharpe"], v["deflated_sharpe"], v["verdict"], v["explanation"])
ci = bootstrap_dsr_ci(returns, n_trials=200)
print(ci["lower"], ci["point"], ci["upper"])
Surface
| Function | Answers |
|---|---|
sharpe_ratio(returns) |
observed per-period Sharpe |
moments(returns, target=0.0) |
mean / std / skew / kurtosis / downside deviation / Sortino |
probabilistic_sharpe_ratio(returns, sr_benchmark=0.0) |
PSR: one minus the p-value of H0: SR <= benchmark (not the probability the true Sharpe exceeds it). sr_benchmark is per period |
deflated_sharpe_ratio(returns, n_trials, trials_sr_std=None, periods_per_year=None) |
PSR against the best of n_trials zero-skill trials (DSR): one minus a p-value, not the probability of skill. trials_sr_std is per period; omitted, it is the annualized 0.5 prior over sqrt(periods_per_year) (default 252) |
expected_max_sharpe(trials_sr_std, n_trials) |
the per-period Sharpe the best of n_trials shows with zero skill, from a per-period trials_sr_std |
min_track_record_length(returns, ...) |
periods needed before the Sharpe is believable; sr_benchmark is per period |
bootstrap_dsr_ci(returns, n_trials, ...) |
{point, se, lower, upper} on the DSR itself; trials_sr_std / periods_per_year as in deflated_sharpe_ratio |
bootstrap_pvalue(excess, ...) |
stationary-bootstrap p-value for one series |
is_my_sharpe_real(returns, n_trials=1, ..., periods_per_year=None) |
LITE verdict dict: pass | borderline | fail + explanation. trials_sr_std and sr_benchmark are annualized and periods_per_year (default 252, flagged) converts both |
is_my_sharpe_real_full(field, ...) |
FULL verdict over a whole candidate field (LITE + snooping family + PBO + HLZ) |
reality_check_pvalue(field, ...) |
White's Reality Check over the field |
spa_pvalue / spa_consistent_pvalue(field, ...) |
Hansen's SPA (liberal / consistent) |
step_down_significant(field, ..., alpha=0.05) |
Romano-Wolf step-down, per candidate, FWER-controlled |
probability_of_backtest_overfitting(perf_matrix, s=16) |
CSCV PBO |
benjamini_hochberg(p_values, q=0.05) / fdr_verdict(...) |
BH-FDR rejections and the operator summary |
hlz_gate(t_stat, t_threshold=None) |
the Harvey-Liu-Zhu |t| >= 3.0 factor bar |
selection_robustness(candidates, n_trials, ...) |
best vs median DSR: is the headline a lucky pick? trials_sr_std / periods_per_year as in deflated_sharpe_ratio |
runs_for_power(effect, alpha, power) |
how many runs to detect an effect |
pass_k(passed_per_run, mode="all", n=None) |
pass^k reliability: won on every run, not on average |
budget_curve(...) |
DSR by search budget, marginal DSR, and the non-improvement onset. trials_sr_std is annualized (default 0.5) and converted by periods_per_year (default 252) |
rank_board(submissions, config_json="") / score_one(...) |
Full composite scoring over the CLI-compatible JSON contract |
rank_returns(field, config_json="") |
Build and rank a board from agent IDs and per-run return arrays |
default_score_config() |
Serialize the default scoring configuration |
never_catastrophic_config() |
Serialize the preset that asks only whether every run avoids catastrophe |
relative_to_benchmark_config(id) |
Serialize the benchmark-relative pass preset |
Units of the deflation prior
Every Sharpe here is per period. The verdicts (is_my_sharpe_real*) and
budget_curve take trials_sr_std annualized, like the leaderboard's
ScoreConfig, and divide it by sqrt(periods_per_year). The verdicts take
their PSR and MinTRL sr_benchmark annualized too and convert it the same way;
through 0.21.0 they read it per period. The raw probabilistic_sharpe_ratio
and min_track_record_length take sr_benchmark per period. The raw primitives
(deflated_sharpe_ratio, bootstrap_dsr_ci, selection_robustness,
expected_max_sharpe) take it per period and use an explicit value as
given. Omitted, the first three use the annualized 0.5 prior at
periods_per_year (default 252): 0.5 / sqrt(252) = 0.0315 per period on daily
returns. Through 0.19.0 they used 0.5 per period, an annualized dispersion of about
7.9 on daily returns.
deflated_sharpe_ratio(returns, n_trials=200) # 0.5 / sqrt(252) per period
deflated_sharpe_ratio(returns, n_trials=200, periods_per_year=8760) # hourly: 0.5 / sqrt(8760)
deflated_sharpe_ratio(returns, n_trials=200, trials_sr_std=0.02) # your per-period value, as given
Matrix orientation
Two conventions, deliberately unchanged from the papers they come from:
- the data-snooping family (
reality_check_pvalue,spa_*,step_down_significant,is_my_sharpe_real_full) takes a field: N rows (strategies) x T cols (time); probability_of_backtest_overfittingtakes the transpose: T rows (time) x N cols (strategies).
Determinism
No I/O, no clock, no ambient randomness. The bootstraps take an explicit seed
(defaulted to a fixed constant, so a result is reproducible unless you ask for
otherwise). The Python suite matches the public example board to the committed
Rust golden on Ubuntu CI. Rust CI separately checks two committed golden fields
on Linux, macOS, and Windows; neither check covers every possible input or host.
Relationship to sharpearena
sharpearena is the environment: a point-in-time market API where a
trading agent produces a track, scored end-to-end by its score_run helper. It
does not provide process containment. sharpebench
is the judge for a track you already have: your own backtest, live P&L, or a
field of candidate strategies. They share one Rust statistics kernel, so the
verdict is identical either way; this package simply does not, and will not,
duplicate arena run-scoring.
Building from source
python -m pip install maturin
python -m maturin develop --manifest-path crates/sharpebench-py/Cargo.toml
python -m pytest crates/sharpebench-py/tests
MIT OR Apache-2.0.
1 maintainer
Metadata
1 maintainer
Release files for sharpebench 0.24.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| sharpebench-0.24.0.tar.gz | 392.1 kB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| sharpebench-0.24.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.13 | CPython 3.13 | Linux glibc 2.17+ x86-64 | Details |
| sharpebench-0.24.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.12 | CPython 3.12 | Linux glibc 2.17+ x86-64 | Details |
| sharpebench-0.24.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.11 | CPython 3.11 | Linux glibc 2.17+ x86-64 | Details |
| sharpebench-0.24.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.10 | CPython 3.10 | Linux glibc 2.17+ x86-64 | Details |
Total release size: 3.1 MB
Release files / sharpebench-0.24.0.tar.gz
| Download URL | sharpebench-0.24.0.tar.gz |
|---|---|
| Size | 392.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f40fd3ece0f5d9c89a71c00d7648cc460f344cf7fffe6a08cd53b6a730f73950
|
|
BLAKE2b-256 checksum How to use checksums |
089783909fe5e11b5085f148b54bf055898e7955da89192094550a1f1bd382ea
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logRelease files / sharpebench-0.24.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | sharpebench-0.24.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 668.7 kB |
| Tags | CPython 3.13 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
41efe997cf44ad3697679c8eacd2351e04e955bf28bc8537bfd4f908ea582b83
|
|
BLAKE2b-256 checksum How to use checksums |
e7b60f71a1f40a6ada406c20bfab0221e3d861e5a7cc4e617f3463c7531c66cd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logRelease files / sharpebench-0.24.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | sharpebench-0.24.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 669.7 kB |
| Tags | CPython 3.12 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
4cc3b6fff86fd4851871833b7434b6698139e5cd2dcc6b146cda88c04646f7a8
|
|
BLAKE2b-256 checksum How to use checksums |
be9eb41094e0967c97e4b3a6f1b8c3c689ae1b23a1e2220d55ae8481ad698cbd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logRelease files / sharpebench-0.24.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | sharpebench-0.24.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 674.3 kB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
11dc161a34b96908b3a317b34eab37b961caefc43b36f07f82198d70a567494b
|
|
BLAKE2b-256 checksum How to use checksums |
76dfa2276118bd12ac8fdc06caccb30634d94e3801722e00b6c8af1ff79490c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logRelease files / sharpebench-0.24.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | sharpebench-0.24.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 674.3 kB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
03f825fbdd9cf256df2022c39dd2d9e44d108d04aab7c40787d6e2d87e598fc6
|
|
BLAKE2b-256 checksum How to use checksums |
68fd3fdf6cb32057084d34dc03997a2a1524f222d2720e2aa9c5a477c88f04b9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency log
