Sakata

Pre-alpha research software. No API stability, no warranty, not for trading. Every interface here can change without notice until 1.0.0.

Sakata is a schema for financial time-series corpora and a way to build one reproducibly. A corpus is defined by a committed text file, not by whatever ended up on disk: delete the data, rebuild from the lock, and every file hashes the same.

Version 0.2.0 adds the data layer — two sources, a content-addressed cache, and the manifest that pins a corpus. There is still no model.

What this is not

• Not a trading system. There is no order routing, no broker integration, and no live market data.
• Not a backtester.
• Not a model. The tokenizer, the architecture, and the forecasting API arrive in v0.4.0 through v0.7.0.

Install

Development only — the repository is private and nothing is published.

uv sync --all-extras

CN A-shares need the optional cn extra; the core install stays numpy and pyarrow.

Building a corpus

Three commands, and the lock is the artefact worth committing.

sakata data resolve examples/crypto-1h-dev.yaml -o corpus.lock.json
sakata data pull corpus.lock.json --dry-run   # file count and byte total first
sakata data pull corpus.lock.json
sakata data verify corpus.lock.json
sakata data stats corpus.lock.json            # read the anomalies

# corpus.yaml
name: crypto-1h-dev
sources:
  - source_id: binance-spot-dumps
    frequencies: [1h]
    start: 2019-01
    end: 2024-12
    universe:
      include_delisted: true
      filters:
        quote_assets: USDT
        min_quote_volume_24h: 1_000_000

Commit the lock. Never the data.

Two sources ship: Binance spot archives and Baostock CN A-shares. Both record their terms in a provenance registry, and neither can be loaded without one — sakata data sources prints the table.

See docs/corpus.md for what each command does, and docs/data-sources.md for what each source does that will surprise you.

Using the schema

from datetime import datetime, timezone
from sakata import BarFrame, Frequency

frame = BarFrame.from_arrays(
    timestamp=[datetime(2024, 1, 1, h, tzinfo=timezone.utc) for h in range(3)],
    venue="BINANCE",
    symbol="BTCUSDT",
    frequency=Frequency.H1,
    open=[100.0, 101.0, 102.0],
    high=[103.0, 104.0, 105.0],
    low=[99.0, 100.0, 101.0],
    close=[101.0, 102.0, 103.0],
)
frame.to_parquet("btcusdt_h1.parquet")
print(frame.validate())  # ValidationReport(ok, 3 rows)
print(frame.close.mean())  # zero-copy numpy view

The conventions worth knowing

A bar's timestamp is its open time, in UTC. Not the close. Choosing close would mean converting on ingestion for every source, and a conversion applied inconsistently is how a lookahead bug gets into a backtest. Naive datetimes are rejected rather than assumed to be UTC.

An instrument is venue plus symbol, as two fields. AAPL trades on several venues, and BTCUSDT differs between Binance and Bybit in both price and volume. The venue is never encoded into the symbol string.

Delisted instruments are included. Binance retains delisted pairs in exchangeInfo, so a survivorship-free universe costs nothing here — 243 of 733 USDT pairs at the time of writing. The liquidity floor deliberately skips them, because a delisted pair reports no current volume and a floor applied to it would exclude every one and quietly undo the correction.

Prices are stored unadjusted. Corporate-action factors are captured beside the bars and never applied. Adjusting at ingest is irreversible, so a policy change would otherwise mean downloading the corpus again.

All four are recorded with their reasoning in docs/decisions.md, along with the six other decisions that are expensive to reverse.

Development

See CONTRIBUTING.md. In short: uv sync --all-extras, uv run pre-commit install, then uv run pytest.

Licence

MIT. See LICENSE and NOTICE — this project derives from Kronos.