polars-hist-db
A Python library for building bitemporal data pipelines with Polars, MariaDB, and optional XTDB backend support.
It ingests data from DSV files or NATS JetStream subjects, tracks history through a typed temporal table contract, and exposes everything as strongly-typed Polars DataFrames.
Features
- Typed uploads — push Polars DataFrames into MariaDB or XTDB with automatic type mapping between Polars, SQL, and SQLAlchemy/ADBC.
- Typed queries — read tables back into DataFrames with column types inferred from the configured backend schema. Temporal query hints (
asof,span,all) let you slice history without writing SQL. - YAML-driven pipelines — define scrape specifications that handle column typing, enrichment via custom transform functions, normalization across tables, and foreign-key deduction.
- Deduplication — an audit log tracks what has already been ingested so re-runs skip previously processed files or messages.
- Transactional ingestion — each file or message is processed in its own transaction; failures roll back cleanly without affecting other items.
- Dual input sources — crawl directories for DSV files, or consume messages from NATS JetStream with configurable fetch/subscription strategies.
- Delta upserts — staging tables detect changed rows, handle duplicates (
take_first,take_last,error), and optionally mark disappeared rows as dropped.
Full documentation: jr200.github.io/polars-hist-db
Quick Start
uv sync
make test
Development
make check # ruff + mypy
make docs # render and preview quarto docs
make bump PART=patch
make release
License
This project is licensed under the MIT license.
1 maintainer
Metadata
1 maintainer
Release files for polars-hist-db 0.12.80
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| polars_hist_db-0.12.80.tar.gz | 980.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| polars_hist_db-0.12.80-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.2 MB
Release files / polars_hist_db-0.12.80.tar.gz
| Download URL | polars_hist_db-0.12.80.tar.gz |
|---|---|
| Size | 980.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
19b51f9b158e16f97379068f165b4bc1d2ef70d235ee62bf93cf26189c459487
|
|
BLAKE2b-256 checksum How to use checksums |
a37dd66499409183f3cbfd1b62f5773db9f81216eee220af843db825126ae593
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / polars_hist_db-0.12.80-py3-none-any.whl
| Download URL | polars_hist_db-0.12.80-py3-none-any.whl |
|---|---|
| Size | 176.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f57769766b82c0e2ff57c45ba496e824d7a9c4ed92543c412ff8954a0523e1c9
|
|
BLAKE2b-256 checksum How to use checksums |
fae14a5c0a94688f019bbc7a1403ca438d512dac189153c826108c9d58943a84
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
