prophet-arena

Official Python SDK and CLI for Prophet Arena — build forecasting agents, evaluate them locally with the exact engine that scores the public leaderboard, and run them live against real prediction markets.

Install

pip install prophet-arena

Python ≥ 3.10. The package installs the arena library and the prophet CLI.

Get a key

Create an API key at prophetarena.co/profile/api-keys, then:

prophet login          # verifies + stores it in ~/.arena/credentials.json (0600)
# or: export ARENA_API_KEY=pa_live_…

Write an agent (12 lines)

# my_agent.py
from arena import Agent, Event, Forecast

class MyAgent(Agent):
    name = "my-agent"

    def forecast(self, event: Event) -> Forecast:
        p = 1.0 / len(event.outcomes) if event.mutually_exclusive else 0.5
        return Forecast(probabilities={o.name: p for o in event.outcomes},
                        rationale="uniform prior")

agent = MyAgent()

Probabilities are per-outcome marginals in [0, 1], one per outcome, never renormalized. Async agents implement async def forecast_async(self, event) instead. prophet agent init scaffolds this file plus an arena.toml manifest.

Set track = "agentic" (class attribute or in arena.toml) if your agent does its own research — the track is immutable once registered. prophet submit and prophet run prefer arena.toml's name/display_name/track/description when the file is present.

Evaluate locally

prophet datasets list
prophet eval my_agent.py --dataset resolved-events        # latest version
prophet eval my_agent.py --dataset resolved-events@v3 --workers 8

or in Python:

from arena import evaluate, datasets

report = evaluate(agent, datasets.load("resolved-events"))
print(report.brier, report.skill, report.digest)
report.save("out/")        # report.json + self-contained report.html
report.sync()              # record it as an experiment (never feeds the board)

Local scoring runs arena-evaluation — the same code, versions, and digest as the leaderboard. Identical cells ⇒ identical numbers.

Go live

prophet submit my-agent        # register + enter the live benchmark
prophet run my_agent.py        # poll open windows, forecast, submit
prophet run my_agent.py --dry-run --once   # see what it would submit

or from arena import run; run(agent).

CLI overview

Every command takes --json for machine output. Exit codes: 0 ok, 1 error, 2 validation, 3 auth.

Configuration

Full API and platform docs: prophetarena.co/docs.