rwpytools

The official Python client for the Robot Wealth research data API — the Python equivalent of rwRTools.

rw-api publishes most datasets two ways: a bulk export (feather / parquet / csv, downloaded through a short-lived signed Cloud CDN URL) and a live endpoint backed by the PostgreSQL warehouse, which always holds the newest rows. The export is refreshed on a schedule; anything after its last date — its watermark — exists only behind the live endpoint.

This library makes that split disappear. client.get(...) downloads the export once, caches it, and on every later call reads it from disk and tops it up from the live endpoint with only the rows newer than the watermark — the diff — joining the two into one DataFrame. And it tells you exactly what it did.

import rwpytools

client = rwpytools.Client("...")

result = client.get("fx_daily", symbols=["EURUSD", "AUDUSD"])

print(result.describe_route())
# fx_daily [-inf .. now] -> bulk[-inf .. 2026-09-23] (cache) live[2026-09-23 .. now]
#   (split at 2026-09-23) — cached export ends 2026-09-23; bulk through the watermark,
#   live diff from 2026-09-23 | 15191 bulk + 0 live = 15191 rows; bulk served from cache

df = result.df

Runnable scripts for common tasks are in examples/.

Install

pip install rwpytools

Python 3.10+. Depends on httpx, pandas, pyarrow, pydantic and platformdirs.

Authentication

client = rwpytools.Client("...")          # explicit
client = rwpytools.Client()               # RWPYTOOLS_API_KEY

Resolution order: constructor argument → RWPYTOOLS_API_KEY → a masked stdin prompt (only in a notebook or a TTY; prompt_for_api_key=False or RWPYTOOLS_NO_PROMPT=1 disables it). A key entered at the prompt is kept in RWPYTOOLS_API_KEY for the life of the kernel, so re-running a cell does not re-prompt. The endpoint defaults to https://api.robotwealth.com and can be overridden with base_url= or RWPYTOOLS_API_BASE_URL.

rw-api reads the key from the api_key query parameter. The library redacts it from every log record and masks it in every repr.

The routing decision

client.get() is the recommended entry point.

The live half starts on the watermark day (hourly exports can end mid-day). Rows both sides return for that seam day are kept once, with the live values: an export's newest day can be provisional (the StatArb price export has carried half-day volumes for its latest date), while the live route holds the settled numbers. Timestamps are matched to the second, absorbing the millisecond jitter in Binance funding times. Live rows are renamed and typed to the export's schema — the Binance export's Ticker / Datetime / Open and the live route's ticker / date / open come back as one rectangular frame. Export-only columns (e.g. Number of trades) are NaN on live rows.

result = client.get("vix", start="2020-01-01")

result.route.sources        # ('bulk', 'live')
result.route.split_at       # datetime.date(2026, 9, 24)
result.watermark            # Timestamp('2026-09-24 00:00:00')
result.bulk_rows, result.live_rows
result.seam_updates         # seam rows whose values were refreshed from live
result.served_from_cache    # was the export already on disk?
result.warnings             # e.g. the live endpoint failed; export returned alone

Choosing the source yourself

Every read — client.get() and every rwRtools-style pod method — takes source=:

client.get("vix", source="bulk")      # the cached export only (downloaded if missing)
client.get("vix", source="live")      # the live endpoint only
client.get("vix", source="auto")      # the default: export + live diff

client.macro.get_vix_vix3m(source="bulk")
client.crypto.get_binance_spot_1h(source="live", start="2026-09-20", symbols=["BTCUSDT"])
client.fx.get_daily_ohlc(["EURUSD", "GBPUSD"], source="bulk", start="2020-01-01")

client.get("vix", force_refresh=True)  # re-download the export first

Preview the decision without fetching anything (reads only local cache metadata):

print(client.explain("binance_spot_1h", start="2024-01-01"))

Why the cache matters — bandwidth

rw-api charges an export's full size against your key's bandwidth cap (20 GiB) the moment it issues a signed URL — even if the bytes then come from a local cache. So a cached export is used without asking rw-api at all, and kept current from the live endpoint instead of being re-downloaded. Datasets without a live endpoint are re-validated (one signed-URL issue, re-downloaded only if the file changed) once their copy is older than bulk_ttl.

Datasets

for spec in client.datasets():
    print(spec.name, spec.bulk, spec.live and spec.live.path)

Per-symbol exports need symbols= (client.get("fx_daily", symbols=["EURUSD"])); ib_shortstock picks its yearly files from start / end (default: the current year).

Pods (rwRtools parity)

Pod method names mirror the rwRtools function with the <pod>_get_ prefix dropped, and return DataFrames routed exactly like client.get():

client.crypto.get_binance_spot_1h()
client.fx.get_daily_ohlc(["EURUSD", "GBPUSD"])       # Date, Open, High, Low, Close, Volume, Ticker
client.fx.get_policy_rates("USD")
client.macro.get_historical_short_sale(2025)
client.equity_factors.get_statarb_spreads(start="2026-01-01")

rwRtools functions whose data rw-api does not publish (the FTX series, coinmetrics, Zorro asset lists, ...) raise DatasetNotSeededError.

Live endpoints

Every endpoint in the API docs has a method on client.live, returning a DataFrame. Cursor-paginated routes are followed to the last page automatically.

client.status()
client.live.yolo_factors()
client.live.yolo_historical(days=30)
client.live.rpschteroids_weights()
client.live.statarb_daily_top_spreads(gte="2026-09-01", limit=5000)
client.live.statarb_liquid_hourly_top_spreads()
client.live.earnings_recent_surprises(lookback_days=20, tickers=["AAPL", "MSFT"])
client.live.ib_short_stocks(ticker="AAPL", gte="2026-09-01")
client.live.forex_daily(ticker="EURUSD", gte="2026-01-01")
client.live.binance_perps_funding(ticker="BTCUSDT")
client.live.macro_rates(symbol="%IRX")
client.live.equity_tickers(tickers=["AAPL", "NVDA"])

Async

AsyncClient mirrors Client exactly, with coroutines:

async with rwpytools.AsyncClient() as client:
    result = await client.get("statarb_spreads", start="2026-01-01")
    eurusd = await client.fx.get_daily_ohlc_ticker("EURUSD")

Cache

Exports live under the platform cache directory (RWPYTOOLS_CACHE_DIR), at <cache_dir>/<bucket>/<object>, each with a .meta.json sidecar recording its size, fetch time, the rw-api cell it serves, and its watermark once read. Downloads commit via temp file + atomic rename, so an interrupted transfer is never served as a cache hit. Eviction is LRU by fetch time against a 10 GiB budget (RWPYTOOLS_CACHE_MAX_BYTES). The cache is thread-safe within a process; it is not coordinated across processes.

client.cache.total_bytes()
client.cache.entries()
client.cache.clear()
client.download("binance_spot_1h")                 # just make sure it is on disk
client.download("fx_daily", symbol="EURUSD")

Errors

from rwpytools.errors import (
    RwPyToolsError,           # base for everything
    ConfigError,              # bad client construction
    RoutingError,             # unknown dataset, bad source / range, missing symbols=
    AuthenticationError,      # 401 — bad API key
    AuthorizationError,       # 403 — valid key, no access
    BandwidthExceededError,   # 403 — bandwidth cap reached; .limit_gb
    NotFoundError,            # 404
    RateLimitError,           # 429 — .retry_after
    ServerError,              # 5xx
    SignedUrlExpiredError,    # CDN rejected the URL — re-minted internally once
    CacheError,               # unrecoverable on-disk cache state
    DataFormatError,          # a response shape the client does not understand
    DatasetNotSeededError,    # rwRtools function whose data rw-api does not publish
)

GETs are retried with exponential backoff and full jitter on transport errors, 429 and 5xx. rw-api's rate limits are per minute and it sends no Retry-After, so a 429 waits rate_limit_wait seconds per attempt. Every RwApiError carries the request_id sent with the request.

In source="auto", a live endpoint that fails with a 5xx, 429 or transport error does not fail the read: the cached export is returned alone and the failure is recorded in result.warnings (and logged). With source="live" it raises.

Configuration

Every setting takes a constructor argument or an environment variable (argument wins).

client = rwpytools.Client(
    "...",
    cache_dir="/fast/disk/rw-cache",
    cache_max_bytes=100 * 1024**3,
    bulk_ttl=6 * 3600,
    max_retries=5,
)

Development

python -m venv .venv && .venv/bin/pip install -e '.[dev]'
.venv/bin/python -m pytest          # mocked HTTP only; any real network call fails the test
.venv/bin/ruff check src tests && .venv/bin/ruff format --check src tests
.venv/bin/mypy                      # strict, with pandas-stubs

Releasing

.github/workflows/release.yml publishes with PyPI trusted publishing (OIDC — no tokens stored):

• every merge to main → TestPyPI as <version>.dev<run_number>;
• a v* tag → PyPI (a pre-release tag such as v0.2.0rc1 goes to TestPyPI).

To release: bump __version__ in src/rwpytools/_version.py via a PR, then git tag v0.1.0 && git push origin v0.1.0 and approve the pypi environment. The one-time PyPI / TestPyPI setup is described at the top of release.yml.

License

MIT. See LICENSE.