Skip to main content

hindsightdb

Capture PostgreSQL row changes in Hindsight for per-field history, blame, and time travel.

Choose a capture mode

Mode Best when What it sees
orm One Python app owns every write SQLAlchemy ORM operations in that process
outbox More than one writer, raw SQL, jobs, or manual fixes touch the database Every committed insert, update, and delete through PostgreSQL triggers

orm is the smallest installation. outbox is the safer operational default because it captures writes regardless of which application made them. It requires a migration and a small shipper in the backend process.

The mode registered in the Hindsight console records the expected capture strategy. It selects setup instructions and health checks; it is not an ingest firewall. If a source is registered as orm but sends outbox events, the events are accepted. The console warns about the mismatch and follows the observed producer for live setup and checks.

Outbox quick start

uv add "hindsightdb[outbox,yaml]"
# or: pip install "hindsightdb[outbox,yaml]"
# capture.yaml
mode: outbox

Set the source key minted by Hindsight and the server URL in the backend runtime:

HINDSIGHT_KEY=hs_your_source_...
HINDSIGHT_URL=https://hindsightdb.com

Start the shipper where the application creates its SQLAlchemy engine:

import os
import hindsight

hindsight.init(
    engine,
    key=os.environ["HINDSIGHT_KEY"],
    mode="outbox",
)

Generate the trigger kit as an application migration:

uv run hindsight apply \
  --dsn "$DATABASE_URL" \
  --config capture.yaml \
  --emit-migration alembic/versions/0042_hindsight.py \
  --down-revision 0041

The migration creates capture_outbox, capture_state, capture functions, and one trigger per captured table. Run it through the application's normal migration pipeline. Do not use automatic installation in a production app that already owns schema changes with Alembic.

If HINDSIGHT_KEY is absent after the migration, application writes still succeed and the triggers still queue changes locally, but nothing drains capture_outbox. Treat the key and shipper as required runtime configuration and monitor outbox depth.

CI and schema drift

Run this against the migrated CI database, after migrations:

uv run hindsight check --dsn "$DATABASE_URL" --config capture.yaml

For outbox capture, the check fails on:

  • a new captured table without a trigger;
  • a missing or disabled trigger;
  • drift in ignored columns, primary keys, tenant columns, or fail-open/fail-closed policy;
  • excluded tables whose old trigger is still installed;
  • missing primary keys, unexplained exclusions, and captured secret-looking columns.

Use the database catalog (--dsn without --metadata) as the production gate when the database contains legacy or unmapped tables. An ordinary new captured column needs no new trigger: the trigger reads the row dynamically. A new table or a policy change does require a regenerated migration, and the check makes that omission fail CI.

The Hindsight console shows the manifest reported by the SDK, last contact, lag, and outbox health. It cannot query the private source database's PostgreSQL catalog, so it complements this CI gate rather than replacing it.

Load the existing rows once

Live capture starts recording new changes immediately. Genesis is separate: it establishes the current state of rows that existed before capture was installed. Run it once inside the backend app container or as a one-off job where both the source database credentials and the Hindsight source key are available:

uv run hindsight genesis \
  --dsn "$DATABASE_URL" \
  --config capture.yaml \
  --run-id your-source-initial

Some platforms expose DB_HOST, DB_PORT, DB_DATABASE, DB_USER, and DB_PASSWORD instead of DATABASE_URL. Build the DSN with the application's existing database helper or pass a complete PostgreSQL DSN to --dsn.

For a busy production database, a consistent dump avoids scanning a changing live snapshot:

uv run hindsight genesis \
  --from-dump source.dump \
  --scratch-dsn "$DATABASE_ADMIN_URL" \
  --config capture.yaml \
  --run-id your-source-initial

Keep the same --run-id when retrying; the server deduplicates that run. Genesis should not run in normal CI, after every migration, or at every application start. Those environments either contain no real baseline or may run several replicas and resend a large database. Make it an explicit, default-off release job if your deployment platform supports one.

ORM quick start

uv add "hindsightdb[sqlalchemy,yaml]"
import os
import hindsight

hindsight.init(engine, key=os.environ["HINDSIGHT_KEY"], mode="orm")

ORM mode needs no database migration, but it cannot see raw SQL, another service, database jobs, or manual changes. Move to outbox mode when that limitation is no longer true.

Metadata

Release files for hindsightdb 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for hindsightdb 0.1.1
File Size Uploaded Provenance Actions
hindsightdb-0.1.1.tar.gz 99.5 kB Without provenance Details for hindsightdb-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for hindsightdb 0.1.1
File Interpreter ABI Platform Actions
hindsightdb-0.1.1-py3-none-any.whl Python 3 none any Details for hindsightdb-0.1.1-py3-none-any.whl

Total release size: 176.7 kB

Release files / hindsightdb-0.1.1.tar.gz

Download URL hindsightdb-0.1.1.tar.gz
Size 99.5 kB
Tags Source
SHA-256 checksum
How to use checksums
9a05d6c883fe336dbf68cfea47752aad86db2497be0d1375af85b8446e34fa5d
BLAKE2b-256 checksum
How to use checksums
448ec5dca97c66d8893ba3621fdd5104c0c402418e7e5ab7280d52d0aad7750f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.3

Release files / hindsightdb-0.1.1-py3-none-any.whl

Download URL hindsightdb-0.1.1-py3-none-any.whl
Size 77.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6c4d5c7092da91d334ffb00f59ecbefca05fca12bdf22e46c3f731f2782443e8
BLAKE2b-256 checksum
How to use checksums
5bea0cee239b419e61ea3c77d1366f8a5f870c040ddec021bbef82a7a8388361
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.3

Security for hindsightdb 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page