timeseries-cache

A persistent SQLite cache for time-series API data. It fetches only missing UTC date ranges and reuses saved rows across runs. The package itself uses only the Python standard library.

Install

Requires Python 3.10 or newer.

uv add timeseries-cache

Use the package

Declare a Schema with one timezone-aware datetime dimension, then create a Cache and call load(selection, start, end, fetcher). The fetcher receives (start, end, refresh) and returns rows with the declared dimensions and a value. load returns rows, a status (live, cache, stale, or unavailable), and any missing ranges. Pass refresh=True to fetch a range again. Cache files are stored in the directory passed to Cache.

Here is a runnable example using generated hourly observations. Replace the fetcher's body with your API call and return rows in the same format.

from datetime import datetime, timedelta, timezone

from timeseries_cache import Cache, Schema

schema = Schema({"timestamp": datetime, "sensor": str}, "timestamp")
cache = Cache("data/cache", schema)
selection = {"sensor": "A"}


def fetch(start, end, refresh):
    timestamp = start
    while timestamp < end:
        yield {"timestamp": timestamp, **selection, "value": 12.5}
        timestamp += timedelta(hours=1)


start = datetime(2024, 1, 1, tzinfo=timezone.utc)
end = start + timedelta(days=1)

first = cache.load(selection, start, end, fetch)
second = cache.load(selection, start, end, fetch)
print(first.status, second.status, len(second.rows))
# With an empty cache: live cache 24

The file data/cache/timeseries_cache.sqlite persists across Python sessions. Requesting a larger interval fetches only the uncovered parts. read() returns stored rows without fetching; missing() returns uncovered intervals.

Always inspect result.error and result.missing_ranges when a load is not successful. A failed fetch can still return previously cached rows.

Cache contract

• One selection fixes every non-time dimension. For multiple series, call load for each selection.
• A fetcher must return the complete response for [start, end), including all pages/chunks, or raise. An empty iterable means the interval was checked successfully. Missing observations alone cannot reveal a failed download.
• Every row contains exactly the schema dimensions and value. Timestamps must be timezone-aware. Missing values use None; NaN and infinity are rejected.
• Include inputs that affect results in options, such as provider URL, aggregation or model version. Use separate directories for separate accounts. The cache cannot identify which API a callback uses. JSON dictionaries must have string keys at every nesting level.
• refresh=True reloads the whole interval. max_age=timedelta(hours=1) reloads only missing or expired intervals. Either option passes True to the fetcher when fetching, allowing it to bypass upstream caches.
• If fetching cannot proceed, covered data returns stale; uncovered intervals return unavailable. missing_ranges describes absent coverage, independent of age. Successful zero-row responses still count as coverage.

By default, SQLite stores observations in cache_rows and fetched intervals in cache_ranges. Setting Schema.partition_dimension creates separate pairs of tables for each selected partition value.

Development

uv sync --extra test
uv run --extra test python -m pytest -q
uv run ruff check .
uv run ruff format --check .

Tests use temporary databases and mocked fetchers. No credentials or live API requests are needed.

Build the wheel and source archive with:

uv build --no-sources