timeseries-table-format (Python)

Build local, append-only time-series tables from Parquet files. The Python API tracks coverage, rejects overlapping appends, and queries tables with DataFusion SQL. Query results are returned as pyarrow.Table objects.

• Documentation
• PyPI
• Repository
• Changelog

Early MVP: APIs and on-disk layouts may change before v1.0.

What it provides

• Timestamp, Int64, or UInt64 ordered indexes
• Per-entity index-interval uniqueness within and across appends
• Local table roots with managed metadata and table-relative Parquet segment paths
• Dry-run and apply vacuum modes for expired unreferenced table-managed files
• DataFusion SQL over managed tables and standalone Parquet data
• Materialized and streaming Arrow result APIs

The current release supports local filesystems only. It does not yet support object storage, small-file compaction, schema evolution, row updates, merges, or time-travel queries.

Install

pip install timeseries-table-format

The package requires Python 3.10 or later and installs pyarrow automatically. See Installation for supported platforms, Python implementations, and source-build guidance.

Verify the installation

import timeseries_table_format as ttf

result = ttf.Session().sql("SELECT 1 AS value")
assert result.column("value").to_pylist() == [1]

Create, append, and query a table

Assuming prices.parquet contains a Timestamp column named ts and a string column named symbol:

import pyarrow as pa
import pyarrow.parquet as pq
import timeseries_table_format as ttf

table = ttf.TimeSeriesTable.create(
    table_root="prices",
    index_column="ts",
    index_type="timestamp",
    index_granularity="1h",
    entity_columns=["symbol"],
)
parquet_file = pq.ParquetFile("prices.parquet")
report = table.append(
    pa.RecordBatchReader.from_batches(
        parquet_file.schema_arrow,
        parquet_file.iter_batches(),
    )
)
version = report.committed_version

session = ttf.Session()
session.register_tstable("prices", "prices")
result = session.sql("SELECT * FROM prices ORDER BY symbol, ts")

For a complete runnable example, follow Create, append, and query your first table.

Documentation

• Decide if the project fits your workload
• Append files incrementally
• Register and join tables
• Use SQL parameters
• Stream query results
• Configure native logging
• Configure notebook display
• Read the API reference
• Troubleshoot common problems

Contributing

See Develop the Python package for local setup, tests, quality checks, documentation builds, benchmarks, and release guidance.

License

MIT. See the repository license.