tradearena-benchmark 0.1.2

Auditable benchmark for LLM trading agents with realistic execution, risk gates, and replayable trajectories.

Open-source benchmark and audit framework for evaluating LLM trading agents under realistic execution, risk, and replayability constraints.

Getting started | Project site | Benchmark card | Submit results | Demo matrix | Contribute | Security

TradeArena

TradeArena turns every trading-agent decision into a traceable trajectory:

observation -> signal -> intended allocation -> risk gate -> order
  -> fill/rejection -> portfolio state -> diagnostic report

It is not another "LLM trading bot." It is a framework for asking whether an LLM trading agent can be audited, reproduced, stress-tested, and constrained before anyone trusts its headline return.

Quick Start

python -m pip install -e ".[dev]"
python scripts/run_showcase.py

Open:

outputs/examples/index.html

The first-run path uses deterministic agents, tracked snapshots, and local demo artifacts. It does not call DeepSeek, Poe, OpenAI, Hugging Face, AkShare, Yahoo Finance, or broker APIs unless you opt into advanced experiments.

No local install yet?

Install And Run

From a clone:

python -m pip install -e ".[dev]"
tradearena --benchmark tradearena-core
python -m tradearena.cli --benchmark tradearena-core

From GitHub without cloning first:

python -m pip install "git+https://github.com/weich97/TradeArena.git"
tradearena --benchmark tradearena-core

PyPI package status: the tradearena distribution name is already occupied by an unrelated project, so the release distribution is tradearena-benchmark. Install from PyPI with:

python -m pip install tradearena-benchmark
tradearena --benchmark tradearena-core

The import namespace and CLI remain tradearena. The historical implementation package trading_agent_os remains available as a compatibility namespace.

Benchmark Result

The v0.1 benchmark card makes one compact claim:

LLM trading-agent evaluation changes materially once intended allocations pass through auditable risk gates and realistic execution constraints.

Open:

• Static page: weich97.github.io/TradeArena/benchmark-v0.1.html
• Markdown artifact: docs/results/benchmark_v0_1.md
• Community registry: docs/results/community_registry.md

Rebuild:

python scripts/build_benchmark_page.py
python scripts/build_benchmark_registry.py examples/benchmark_submissions

Submit Or Validate A Benchmark Row

TradeArena supports redacted benchmark submissions. They share scenario, execution, risk, metrics, and reproducibility metadata without exposing raw provider prompts, responses, credentials, or private portfolios.

tradearena validate-submission examples/benchmark_submissions/example_redacted_submission.json
tradearena build-registry examples/benchmark_submissions --output docs/results/community_registry.md
tradearena hash-run outputs/examples/audit_walkthrough_trajectory.json

See docs/benchmark_submissions.md.

Visual Preview

Audit lifecycle	Execution realism	Diagnostic loop

The browser-playable launch video is here: weich97.github.io/TradeArena/demo_video.html.

What TradeArena Provides

Need	TradeArena surface
Replayable decisions	Trajectory logs with prompts, memory digests, risk reports, fills, and metrics
Execution realism	Fees, spread, slippage, latency, liquidity caps, partial fills, and rejections
Risk-aware evaluation	Pre-trade gates, in-trade monitors, post-trade attribution, violations
Extensibility	Data, analyst, strategy, risk, simulator, memory, planner, evaluator plugins
Community benchmarks	Redacted submission schema, registry builder, reproducibility hashes

Extension Path

Start with one small plugin:

python examples/custom_plugin_demo.py
python examples/extension_walkthrough_demo.py

The walkthrough swaps in a custom analyst, risk manager, and evaluator while reusing the existing runner, data provider, strategy, execution simulator, memory store, trajectory logger, and metric stack.

Useful entry points:

• examples/README.md
• docs/demo_matrix.md
• docs/extension_walkthrough.md
• docs/contributor_roadmap.md

Documentation Map

• Quickstart: docs/getting_started.md
• Schemas: docs/schemas.md
• Benchmark submissions: docs/benchmark_submissions.md
• Related work: docs/related_work.md
• Retail planning sandbox: docs/retail_planning.md
• Research protocol: docs/research_protocol.md
• Security policy: SECURITY.md
• Governance: GOVERNANCE.md

Local Checks

Each checkout can use its own .venv, so public and private repos do not fight over editable installs:

powershell -ExecutionPolicy Bypass -File scripts\check_local.ps1

The script installs the current checkout in editable mode, runs compile checks, Ruff critical checks, tests, release-readiness checks, submission validation, artifact-contract validation, and JSON validation.

Safety Boundary

TradeArena does not promise profitable trading, does not provide financial advice, and does not execute live trades by default. Public examples are offline, paper-only, or human-review oriented. Broker and provider integrations must follow SECURITY.md and GOVERNANCE.md.

Cite

See CITATION.cff. If you use TradeArena in research or software, cite the repository release you used.