RL-pricing

A reproducibility lab for trading AI. RL-pricing re-tests published claims about deep RL agents, LLM trading agents and forecasters under one pre-registered protocol. This package holds the lab's instruments:

• rl-pricing bench: a causal backtesting benchmark. Policies decide at the close and fill at the next open with fees; learned policies run over several seeds; frozen policies are scored on a live forward-test track; and a decision ledger records LLM agents' trades before each US market open.
• rl-pricing --mode train|backtest|ensemble|lstm: the share-trading toolkit that trains and backtests Stable-Baselines3 agents (A2C, PPO, DDPG, TD3, SAC), a rolling ensemble and an LSTM forecaster on daily equity data.

Score your own policy

A policy maps the history up to today's close to long-only target weights, which fill at the next open. Put it in a submission directory:

# my-submissions/low-vol/policy.py
import numpy as np

from rl.bench.policy import Policy


class LowVolHalf(Policy):
    """Equal-weight the calmer half of the listed assets."""

    def act(self, view):
        closes = view.history.window(64).close
        vol = np.nanstd(np.diff(np.log(closes), axis=0), axis=0)
        vol = np.where(view.listed & np.isfinite(vol), vol, np.inf)
        keep = np.argsort(vol)[: max(1, view.listed.sum() // 2)]
        weights = np.zeros(len(view.tickers))
        weights[keep] = 1 / len(keep)
        return weights

and describe it in my-submissions/low-vol/submission.json:

{"name": "Low-vol half", "kind": "other", "policy": "policy.py:LowVolHalf", "trials": 1}

A submission can also name a built-in policy, such as "policy": "rl.bench.policies.baselines:Momentum" or "policy": "rl.bench.policies.sb3:SB3Policy" with "params": {"algo": "ppo"}. Then score it against 1/N buy-and-hold:

pip install rl-pricing
rl-pricing bench tasks
rl-pricing bench run --task synthetic --submission low-vol \
    --submissions-root my-submissions --out-root board
rl-pricing bench leaderboard --out-root board --no-readme

synthetic runs offline; the real tasks (us-sectors, us-multi-asset, crypto-majors, …) download their data on first use into datasets/bench/, or $RL_PRICING_DATA.

The import package is rl, so this distribution can't share an environment with the unrelated rl (GNU Readline bindings) distribution. The policy interface, the submission.json fields, the record formats and the rl-pricing commands follow semantic versioning; see the changelog.

• Verdicts, findings and leaderboard
• Paper: Wang & Zhang, arXiv:2609.31870

The training toolkit's environment, preprocessing and training code are adapted from FinRL, which is MIT-licensed.