ml-data-access

Read side of the malatium data store. One load_* per table that ml-data-pipelines writes, over a bear-lake database, in the shape of at-research's data helpers.

pip install ml-data-access

import datetime as dt

import ml_data_access

db = ml_data_access.connect()  # ML_DATA_STORE, or connect(path)
start, end = dt.date(2018, 7, 2), dt.date(2025, 6, 30)

reference_df = ml_data_access.load_reference_returns(db, start, end)
scores_df = ml_data_access.load_signals(db, "vrp", start, end)
loadings_df = ml_data_access.load_factor_loadings(db, start, end)
chain_df = ml_data_access.load_option_greeks(db, "AAPL", dt.date(2025, 1, 1), dt.date(2025, 3, 31))

Every loader is load_x(db, start=None, end=None) and returns a collected DataFrame for the inclusive window; the three per-symbol chain loaders take the symbol first. Nothing is screened, joined or derived: the store holds canonical tables (symbol is the option root, right is C/P, iv is null where the vendor's inversion failed, vega is per vol point).

The frames slot straight into malatium:

from malatium.providers import PanelProvider, TradingCalendar

calendar = TradingCalendar(ml_data_access.load_sessions(db, start, end))
reference = PanelProvider(reference_df)
scores = PanelProvider(scores_df)

Loaders

ml_data_access.describe() lists what the store holds. ml_data_access.scan(name, years, symbols) is the lazy scan under every loader; it opens only the partitions asked for, using bear-lake's <table>/<year>/<symbol>.parquet layout.

Things to know before trusting a number

• Eighteen symbol-years are another company's chain. Semi-join against usable_symbol_years(db) for any multi-year study.
• Open interest is one day stale by construction and joins on the same date.
• iv is null, not wrong, on the ~3% of contract-days that failed to invert.
• Repaired index sessions (mostly 2020-21) carry null gamma and a 15:59 underlying.

Development

uv sync
uv run pytest        # builds a synthetic store in a temp dir
uv run ruff check . && uv run ruff format .