OHLC Toolkit

A Polars-native toolkit for reading, validating, and aggregating OHLC (Open, High, Low, Close) market data.

Each stage is a separate, composable step: fetch a published dataset, read and validate it, aggregate windows over it, add returns, apply a quality policy. Nothing sorts, fills, interpolates, or de-duplicates your data on the way past. When the data is not what you expected, you get a value describing that — or an exception — rather than a tidied frame and a log line.

Installation

pip install ohlc-toolkit

Python 3.11 or newer. Seven declared runtime dependencies: polars, requests, loguru and orjson, which this package imports, plus urllib3, idna and certifi, which it does not. Those three are declared because requests carries them into every install and its own ranges admit versions with published advisories against them; a floor here is the only place a consumer's resolve can be closed.

1.0 is a break

1.0 is a rewrite, not an upgrade. Every name 0.4 exported is gone:

There are no aliases, no deprecation warnings, and no migration path. The replacements are not renames — they take different arguments, return different types, and mean different things. A shim would have had to guess which of those differences you wanted, and guessing about a price series is how a wrong number reaches a model.

pandas is no longer a dependency of any kind.

0.4.x stays on PyPI and keeps working. If you depend on it, pin it:

pip install "ohlc-toolkit<1"

Quickstart

Against the real published BTC/USD minute history (ff137/bitstamp-btcusd-minute-data). The release is about 260 MB across three assets; it is fetched once and re-used on later runs. The toolkit logs each fetch and verification step as it runs; those log lines are omitted from the pasted output below.

from ohlc_toolkit.returns import (
    ReturnMethod,
    add_backward_returns,
    add_forward_returns,
)
from ohlc_toolkit.snapshot import (
    BITSTAMP_BTCUSD_1M_REPOSITORY,
    SnapshotRelease,
    fetch_snapshot,
    read_snapshot_frame,
    verify_snapshot_continuity,
)
from ohlc_toolkit.source import BITSTAMP_BTCUSD_1M
from ohlc_toolkit.windows import compute_windows

# Fetch the published minute history. Nothing lands until its SHA-256 and
# size match what the release's own manifest declared.
release = SnapshotRelease(
    repository=BITSTAMP_BTCUSD_1M_REPOSITORY,
    tag="bitstamp-btcusd-1m-2026-08",
)
result = fetch_snapshot(release, "data/bitstamp")
print("snapshot identity:", result.manifest_sha256)

# Read it under strict validation, then check the frame against what the
# manifest said it would be.
frame = read_snapshot_frame(result)
report = verify_snapshot_continuity(frame, result.manifest)
print("rows:", report.rows_checked, "| seam mismatches:", report.seam_mismatches)

# One-hour windows, emitted every fifteen minutes.
hourly = compute_windows(
    frame,
    BITSTAMP_BTCUSD_1M,
    window="1h",
    emit_every="15m",
    materialization="skip_warmup",
)
print(hourly.tail(3))

# The causal four-hour log return, and the non-causal one beside the
# instant it becomes readable.
labelled = add_forward_returns(
    add_backward_returns(
        hourly, horizon="4h", cadence="15m", method=ReturnMethod.LOG
    ),
    horizon="4h",
    cadence="15m",
    method=ReturnMethod.LOG,
)
print(
    labelled.select(
        "close_time",
        "backward_return_log_4h",
        "forward_return_log_4h",
        "forward_return_log_4h_available_at",
    ).tail(3)
)

snapshot identity: 96e96cc32b313e4985a3d2d105e40ee528f8243bd2d1146a38f1b600f0bd3de1
rows: 7714079 | seam mismatches: ()
shape: (3, 10)
┌────────────┬───────────┬──────────┬──────────┬───┬───────────┬───────────┬───────────┬───────────┐
│ open_time  ┆ close_tim ┆ open     ┆ high     ┆ … ┆ volume    ┆ src_count ┆ coverage_ ┆ traded_se │
│ ---        ┆ e         ┆ ---      ┆ ---      ┆   ┆ ---       ┆ ---       ┆ seconds   ┆ conds     │
│ i64        ┆ ---       ┆ f64      ┆ f64      ┆   ┆ f64       ┆ u32       ┆ ---       ┆ ---       │
│            ┆ i64       ┆          ┆          ┆   ┆           ┆           ┆ i64       ┆ i64       │
╞════════════╪═══════════╪══════════╪══════════╪═══╪═══════════╪═══════════╪═══════════╪═══════════╡
│ 1788215400 ┆ 178821900 ┆ 78734.71 ┆ 78742.28 ┆ … ┆ 37.490404 ┆ 60        ┆ 3600      ┆ 3600      │
│            ┆ 0         ┆          ┆          ┆   ┆           ┆           ┆           ┆           │
│ 1788216300 ┆ 178821990 ┆ 78642.61 ┆ 78642.61 ┆ … ┆ 34.778154 ┆ 60        ┆ 3600      ┆ 3600      │
│            ┆ 0         ┆          ┆          ┆   ┆           ┆           ┆           ┆           │
│ 1788217200 ┆ 178822080 ┆ 78572.31 ┆ 78576.84 ┆ … ┆ 36.012591 ┆ 60        ┆ 3600      ┆ 3600      │
│            ┆ 0         ┆          ┆          ┆   ┆           ┆           ┆           ┆           │
└────────────┴───────────┴──────────┴──────────┴───┴───────────┴───────────┴───────────┴───────────┘
shape: (3, 4)
┌────────────┬────────────────────────┬───────────────────────┬─────────────────────────────────┐
│ close_time ┆ backward_return_log_4h ┆ forward_return_log_4h ┆ forward_return_log_4h_availabl… │
│ ---        ┆ ---                    ┆ ---                   ┆ ---                             │
│ i64        ┆ f64                    ┆ f64                   ┆ i64                             │
╞════════════╪════════════════════════╪═══════════════════════╪═════════════════════════════════╡
│ 1788219000 ┆ -0.006804              ┆ null                  ┆ 1788233400                      │
│ 1788219900 ┆ -0.005415              ┆ null                  ┆ 1788234300                      │
│ 1788220800 ┆ -0.003928              ┆ null                  ┆ 1788235200                      │
└────────────┴────────────────────────┴───────────────────────┴─────────────────────────────────┘

Those three forward returns are null because their counterparts lie past the end of the data — and their available_at is stated anyway, because when a value would arrive is a property of the horizon, not of whether it happened to be found.

Timings from that run, on a 16-core desktop: 2.8 s to read and strictly validate all 7,714,079 rows, 0.8 s to aggregate the 514,268 windows, and under 0.1 s for both return columns. polars uses the whole thread pool.

What's in it

import ohlc_toolkit reaches all seven subpackages. Names are not flattened into the top level — spell them ohlc_toolkit.windows.X, or import from the subpackage.

temporal — durations that carry their unit

Duration holds exact whole seconds and parses one compact grammar: components in strictly descending order, each unit at most once, no separators or signs.

from ohlc_toolkit.temporal import Duration

Duration.parse("1h15m").total_seconds == 4500
str(Duration(4500)) == "1h15m"

Anywhere a duration is accepted, a Duration or its string spelling both work; a bare integer whose unit you have to infer does not. ConfigError, DataValidationError and CoverageError are the package's error taxonomy, and every message that quotes untrusted input goes through one bounded echo.

source — reads that report instead of repair

A SourceProfile states a source's cadence, phase, timestamp column, and raw column kinds. read_source_csv reads against one without sorting, filling, dropping or de-duplicating anything, and validate_source_frame returns findings as data:

import polars as pl

from ohlc_toolkit.source import (
    BITSTAMP_BTCUSD_1M,
    ValidationMode,
    validate_source_frame,
)

frame = pl.DataFrame(
    {
        "timestamp": [1786924800, 1786924860, 1786924980],  # 1786924920 missing
        "open": [1.0, 2.0, 3.0],
        "high": [1.0, 2.0, 3.0],
        "low": [1.0, 2.0, 3.0],
        "close": [1.0, 2.0, 3.0],
        "volume": [1.0, 1.0, 1.0],
    }
)
report = validate_source_frame(frame, BITSTAMP_BTCUSD_1M, mode=ValidationMode.REPORT)
for finding in report.findings:
    print(finding)

Finding(kind=<FindingKind.GAP: 'gap'>, message='1 missing candle(s) expected in [1786924920, 1786924980)', count=1, sample_timestamps=(1786924920, 1786924980))

The eight finding kinds are schema, nulls, non-finite values, non-increasing timestamps, overlapping intervals, off-phase timestamps, irregular intervals, and gaps. A non-finite value is a NaN or an infinity in a declared price or volume column; it is kept distinct from a null because a null is an absent cell and a NaN is a present cell that is not a number, and neither is coerced into the other. ValidationMode.STRICT raises SourceValidationError on any of them instead.

windows — aggregation with an independent oracle

compute_windows emits one row per tick of an epoch-anchored emit grid, each row aggregating the candles whose intervals fall inside the window ending at that tick. Membership is decided by close time, never by counting rows, so a gap in the source changes the window's reported coverage rather than silently changing what it spans. Ten columns come back: open_time, close_time, OHLCV, src_count, coverage_seconds, traded_seconds. The last two measure different things: coverage is how much of the window the source had rows for, traded_seconds how much of it those rows traded in. On a complete grid coverage is full everywhere and a dead window looks exactly like a busy one without the second number.

compute_reference_windows computes the same thing the plainest possible way — quadratic, on purpose. On valid input it is the specification and the engine is what you run; neither validates what it is handed, and on a NaN or a null price the two do not agree, which both docstrings state. The suite holds the two to the same rows, in the same order, with the same dtypes, across a synthetic matrix, property tests, committed goldens, and a real 14-day slice — exactly equal on every integer column and every selected price, and within a tolerance derived from the oracle's own fold on volume, the one column either implementation sums. Both resolve schedules through the same module, so a configuration one refuses is refused by the other in the same words.

apply_quality_policy is a separate, later step over the output, never inside it. PASS_THROUGH records a deliberate no-op, FILTER drops rows below a coverage threshold, GATE raises (or reports) without dropping. Its threshold is exact rational arithmetic, not a float product.

annotate_windows is another later step: it joins a sparse sidecar of half-open [start, end) intervals -- an outage log, a maintenance calendar -- onto the output as two appended columns, the distinct flags overlapping each window and the seconds of the window inside the union of those intervals. Overlap is half-open on both sides, so an interval that ends exactly at a window's open touches nothing; flags are opaque strings the transform reports and never interprets; no OHLCV or coverage value is read or changed. read_annotations reads such a sidecar from CSV, typed and checked, in file order and under an optional row cap, with the Bitstamp provenance file's column names as defaults; a broken sidecar raises AnnotationValidationError, a misconfigured call ConfigError.

schedules — schedules that record what they are

Generators (log_spaced, metallic_recurrence, explicit) produce a WindowSchedule; cadence rules (w_over_k, explicit_pairs) produce a CadenceRule. There is no default schedule, coefficient, bound or divisor anywhere: a caller states what it wants and gets back something that records exactly what was asked for, named by a content hash over that record.

from ohlc_toolkit.schedules import WindowSchedule, log_spaced

schedule = log_spaced(count=5, minimum="15m", maximum="1d", grain="15m")
print([str(window) for window in schedule.windows])
print(schedule.schedule_id)
print(WindowSchedule.from_dict(schedule.to_dict()) == schedule)

['15m', '45m', '2h30m', '7h45m', '1d']
0abc992caafb27a7ce8ba1ba5edca1118366b9ec84190744e2a641c0972d3e07
True

A lookback schedule is the count-valued twin: its members are period counts, not durations, so the same 21 is twenty-one minutes on a 1m frame and twenty-one weeks on a 1w one. It is a separate type rather than a setting, because a lookback borrowed from a window schedule would be silently coupled to it. The arithmetic is shared — the same recurrence, the same log-spaced placement, the same quantize/bound/dedup rule — and only the unit differs.

import math

from ohlc_toolkit.schedules import metallic_lookback

lookback = metallic_lookback(
    coefficient=math.sqrt(math.e + math.sqrt(5)),
    seed=1,
    grain=1,
    minimum=3,
    maximum=21,
)
print(lookback.periods)
print(lookback.schedule_id)

(3, 8, 21)
2b7c4fa642bfd96d9727e10ea28438ffcbb7756b9a5809e8e57a56339cc9b0b4

The two identities cannot collide. A lookback records its members under periods and a window schedule under windows, and 3 is not "3m", so {1, 3, 8} and {1m, 3m, 8m} hash differently and each reader refuses the other's payload rather than reading it as its own.

A payload whose recorded schedule_id does not match its content is refused rather than repaired, so a schedule read back from disk is the one that was written.

returns — causal and non-causal, told apart

add_backward_returns relates a row's close to the close exactly H earlier; both were known at the row's own close time, so it is a feature. add_forward_returns relates the close H later to the row's own, and is not. The distinction is carried in the data twice — the forward_ prefix travels with the column name into any file or plot, and every forward value has an available_at column stating the instant it may first be read.

Counterparts are located by exact close-time equality, never by shifting rows, so a gap yields a null instead of a wrong pairing. method is required — ReturnMethod.SIMPLE or ReturnMethod.LOG — and is recorded in the column name. Any value that is not a finite float comes back null.

snapshot — fail-closed fetching

A release is named by repository and immutable tag; asset URLs are composed from that identity rather than discovered, so there is no API call, no token, and no listing step to disagree with the manifest. Downloads land on a temporary name and are renamed into place only after size and SHA-256 both match what the manifest declared. An already present file is reused only when its digest matches, and a mismatch is refused rather than overwritten unless you pass ExistingAssetPolicy.REPLACE.

fetch_snapshot also returns the snapshot identity — the digest over the manifest bytes — and accepts it back as expected_manifest_sha256, which refuses both a wholesale manifest swap and a release re-cut under the same tag.

verify_snapshot_on_disk answers the other question. Fetching asks did these bytes arrive intact; this asks are these still the bytes, which is what a caller reading a snapshot it downloaded last week needs — a directory verified in August is not thereby a directory verified now, and nothing about a directory stops something else writing to it. It touches no network, checks presence, size and digest for every declared asset, and returns the same result type read_snapshot_frame consumes.

What it proves is that the directory is internally consistent: these bytes are the bytes this manifest describes. It does not prove the manifest is the one you meant — a different, self-consistent release verifies clean and is reported as itself, which is the right behaviour, since the identity it returns is what you record.

indicators — the phased lookback every indicator reads through

At each tick of the emit grid, a phased indicator consumes the L non-overlapping windows of duration W ending at t, t − W, t − 2W, and so on. phased_lookback is the fast path; phased_lookback_reference is the brute-force oracle it is tested against.

The inputs are read from the window's source-cadence materialization, not from its own E-cadence frame. {t − kW} lies on the emit grid only when E divides W, and it frequently does not: at W = 2h26m against E = 3m, the emit frame holds none of the phases at all.

Lookups are exact equality on close_time — never a shift, a nearest match or an as-of join. A window whose traded_seconds falls below the recipe's threshold is a null input, applied here whatever quality mode wrote the frame, because report mode removes nothing. Any null among the L inputs nulls the whole output row rather than leaving a list with a hole in it, so no indicator downstream has to remember the rule. The effective history L × W comes back with the result rather than being left to callers to multiply.

A feature column's name is derived from its identity, never passed beside it: {indicator}_{family}{period}_w{window} — rsi_p14_w21m, logvolratio_p14_w2h26m — with the window spelled the same way parquet filenames and manifest fields spell it. What varies within a frame is in the name; what is constant across the artifact is in the manifest, so the two cannot disagree. The name parses back to the record it came from.

Every feature reports two counts. Effective history is L × W, the span one value is computed from. Effective-N counts independent blocks — whole non-overlapping windows in the range, divided by the lookback — and is deliberately not a statistical effective sample size, which accounts for autocorrelation and is smaller. They have different names because reading one as the other overstates the evidence.

Development

git clone https://github.com/visikai/ohlc-toolkit.git
cd ohlc-toolkit

uv sync --all-groups

uv run pytest
uv run mypy .
uv run ruff check .

pytest deselects the network-marked tests by default; run them with uv run pytest -m network to fetch and verify the real published release. benchmarks/window_engine.py measures the window engine over a full minute history against independently computed references.

Support

If you need any help or have any questions, please feel free to open an issue or contact me directly.

We hope this repo makes your life easier! If it does, please give us a star! ⭐