pgnudge
Your database moved. Your app already knows.
pgnudge is a tiny async library that tells you which tables just changed in PostgreSQL, so a live read model can re-render the instant the data moves. It leaves nothing behind on the server - no triggers, no functions, no persistent slots, no cleanup jobs. Close the connection and PostgreSQL forgets pgnudge ever existed.
It carries no row data, by design. You already know how to load your data; pgnudge just tells you when, and what to reload.
from pgnudge import Batch, Resync, WalFeed
async with WalFeed(host="db", user="wal_user", password=..., database="app",
tables=["public.orders", "public.stations"]) as feed:
async for item in feed:
match item:
case Resync(): # (re)connected: reload everything
await reload_everything()
case Batch(events=evs): # coalesced wakeups: which tables moved
await reload(tables={e.payload for e in evs})
pip install pgnudge
Python >= 3.11, PostgreSQL >= 16. One runtime dependency: scramp (pure-Python SCRAM auth). No database driver - pgnudge speaks the PostgreSQL replication protocol itself.
Why pgnudge
- Zero server footprint. The only server object is a temporary
replication slot, dropped automatically the instant the session ends -
clean close, crash,
kill -9, orpg_terminate_backend.RawFeedneeds no slot at all. Nothing to install, migrate, review, or clean up. - Driver-free, one dependency. A hand-rolled walsender client (TLS,
SCRAM-SHA-256, CopyBoth) instead of a database driver.
pip install pgnudgepulls in scramp and nothing else. - Two transports, one contract.
WalFeed(logical decoding) if you can setwal_level = logical;RawFeed(physical WAL, decoded client-side) on a stockwal_level = replica. SameResync | Batchstream either way - the choice is one constructor and touches nothing else. - Coalesced wakeups. A 500-row transaction on one table is one
Event,count=500, one wakeup, one refetch. Debounced client-side. - Correct by construction. At-least-once wakeups from the point of connect;
every gap is bracketed by a
Resync. HandleResyncand nothing can make your view wrong. No cursors to persist, no exactly-once to get wrong. - async-first and typed.
async for item in feed. Strict mypy, 100% line+branch coverage, claims proven against real PostgreSQL in CI. - Preflight
doctor. One command connects, fingerprints the platform (RDS/Aurora, Azure, Cloud SQL, or self-managed), and tells you which feed to use - with a copy-paste fix under every failed check, tailored to that platform. Leaves nothing behind.
Should you use pgnudge?
| Reach for pgnudge when... | Look elsewhere when... |
|---|---|
| A dashboard / cache / read model must re-render on change | You need the changed rows (before/after images) -> that's CDC (Debezium) |
| You can refetch from the DB - it's the source of truth | Every message must be processed exactly once -> use a queue (pgqueuer) |
| You want nothing installed in the database | You need history / backfill of changes that happened while disconnected |
| Missing changes while disconnected is fine (you'll refetch) | You need cross-datacenter durable replication -> use logical replication |
pgqueuer moves work; pgnudge moves wakefulness.
Get started
You make exactly one decision: which feed class. It is driven by a single
server setting, wal_level.
- You can set
wal_level = logical-> useWalFeed. The fuller transport:TRUNCATEnudges too, the server filters tables for you, and only your database's WAL is decoded. Costs a one-time restart on most servers, plus an output plugin (wal2json, preinstalled on most managed platforms, ortest_decoding, built into PostgreSQL). - You are stuck at the stock
wal_level = replica-> useRawFeed. No server change at all: it decodes physical WAL client-side. The trade: noTRUNCATEdetection, and the server streams the whole cluster's WAL for pgnudge to filter locally.RawFeedis best treated as a self-hosted / VM-Postgres transport: it needs external physical WAL streaming, which managed platforms are not known to expose (untested; see Managed platforms). On a managed service, flipwal_level = logicaland useWalFeed.
If you get to choose, choose WalFeed. Neither is more "correct"; they are the
same contract over two different server capabilities.
Either way you need a role with the REPLICATION attribute and a direct
connection (replication traffic cannot go through a pooler like PgBouncer).
Then:
pip install pgnudgepgnudge doctor --host ... --user ... --database ...- it connects, checkswal_level, theREPLICATIONgrant, and the output plugin, then tells you which feed to use, printing a copy-paste fix under any failed check (tuned to the detected platform: RDS parameter groups,azcommands, or plainALTER SYSTEM). Ifwal2jsonis absent it retries with the built-intest_decodingso it can distinguish "logical decoding is blocked" from "logical works, the plugin just is not installed". The WalFeed check creates a temporary slot and drops it, sodoctorleaves nothing behind.- point that feed at the database (the tour below)
- handle the two items:
Resync-> reload everything,Batch-> reload the named tables
That is the whole setup. Nothing is installed in the database and nothing is left behind when the connection closes.
Sixty-second tour
from pgnudge import Batch, Resync, WalFeed
async with WalFeed(
host="db.example.com", user="wal_user", password=...,
database="app", ssl=True,
tables=["public.orders", "public.stations"], # filtered in the output plugin
debounce=0.05,
) as feed:
async for item in feed:
match item:
case Resync(): # connected / reconnected / overflow / failsafe
await reload_everything()
case Batch(events=evs): # coalesced wakeups: which tables moved
await reload(tables={e.payload for e in evs})
There is no step 1. Nothing to install in the database, nothing to migrate. Close the connection and the server forgets pgnudge ever existed.
The contract
A feed yields exactly two item types:
Resync(reason): reload everything. Emitted on every connect and reconnect, on internal queue overflow, and (optionally) on a failsafe interval. HandleResynccorrectly and nothing can make your view wrong.Batch(events): one debounce window's worth of wakeups, deduplicated, in arrival order. EachEventcarriespayload(schema.table, the stable v1 payload contract),first_seen,count.
Delivery is at-least-once wakeups, from the point of connect only. Events
are hints to refetch, never facts to apply. There is no history and no backfill,
by design and by mechanism: the slot is created fresh at every (re)connect with
SNAPSHOT 'nothing', and a logical slot can only decode forward from its
creation point. The handshake is gap-free. Resync is emitted only after the
stream is live, so the refetch it triggers observes a state at or after the
slot's start point, and every later commit produces a nudge; anything landing in
between is covered twice, which at-least-once absorbs. On reconnect WalFeed
resyncs rather than resumes. No replay, no exactly-once, no row images:
refetching is idempotent and you have a database right there. (One nuance: slot
creation waits for write transactions in flight at connect time, so a
long-running write delays connect, but it never causes history to be delivered.)
Coalescing: per-row changes within the debounce window collapse client-side
into one Event with a count. A 500-row transaction on one table is one
Event, count=500, one wakeup, one refetch.
INSERT, UPDATE, DELETE, and TRUNCATE all nudge on WalFeed (RawFeed
covers all but TRUNCATE). Neither transport carries other DDL, so schema
changes don't nudge; pair migrations with a refetch if your view depends on
them.
The zero-footprint guarantee
WalFeed creates nothing on the server that outlives the connection.
your app ──── async for item in feed ────▶ Resync | Batch
▲
│ walsender protocol, no driver (TLS, SCRAM-SHA-256, CopyBoth)
│
PostgreSQL ── TEMPORARY replication slot ── logical decoding
└── dropped by the server the instant the session ends,
cleanly or not
The temporary replication slot is the only primitive in PostgreSQL that gives
you a change feed with connection-scoped lifetime: the server is contractually
obliged to drop it the moment the session ends, whether by clean close, crash,
kill -9, or pg_terminate_backend. No triggers, no functions, no persistent
slots, no cleanup jobs. The test suite ends by hard-aborting the socket with no
protocol goodbye and asserting pg_replication_slots is empty.
What is required is PostgreSQL 16+ and one-time server configuration
(settings, not objects): wal_level = logical, a role with REPLICATION, and
an output plugin. That plugin is wal2json (default; preinstalled on Azure
Flexible Server, RDS, and most managed platforms) or test_decoding (ships
inside PostgreSQL itself).
The full mechanics (logical decoding, temporary-slot semantics, the gap-free handshake argument, and when not to use pgnudge) are in docs/temporary-slots.md.
Two transports, one contract
Both feeds yield the same Resync | Batch stream; pick by what your server
allows.
WalFeed (logical decoding) |
RawFeed (physical WAL) |
|
|---|---|---|
wal_level |
logical (usually needs a restart) |
replica, the stock default |
| Output plugin | wal2json or test_decoding | none; WAL is decoded client-side |
| Server objects | one TEMPORARY slot while connected | none at any point, not even a slot |
TRUNCATE nudges |
yes | no (documented gap) |
| Stream scope | one database, filtered server-side | whole cluster, filtered client-side |
RawFeed exists for self-hosted servers where wal_level=logical is not on
the table: change-averse ops, no restart window, or a policy against logical
decoding. It needs external physical WAL streaming, which managed platforms are
not known to expose (untested; see Managed platforms), so
on a managed service you use WalFeed. It streams raw physical
WAL slot-less and parses record headers client-side, just enough to answer
which relation changed, never row contents. Nudges are commit-gated: a change
is delivered only after its transaction's commit record, so rollbacks never
nudge and a refetch never races an open transaction. The start position is the
server's current WAL insert point, so from-connect-only holds exactly as it does
for WalFeed.
from pgnudge import RawFeed
feed = RawFeed(
host="db.example.com", user="wal_user", password=..., database="app",
tables=["public.orders", "public.stations"], # client-side filter
)
Costs, stated honestly: the server sends the whole cluster's WAL to the client
(every database, index churn, vacuum traffic); pgnudge filters client-side, but
the bandwidth is paid. TRUNCATE is not detected at wal_level=replica (the
WAL carries no reliable signature for it; the next write to the table nudges
normally). RawFeed also opens a second, plain connection for catalog lookups
(relfilenode to table name), and pg_hba.conf needs a replication entry for
the role, because physical replication matches the replication
pseudo-database, not all. Mechanics in docs/physical-wal.md;
the byte layouts and parser structures behind both transports are in
docs/parsing.md.
Managed platforms
Short version: each platform below documents a WalFeed path - flip
wal_level = logical and grant a REPLICATION-capable role. Whether RawFeed
works (it needs external START_REPLICATION PHYSICAL to a non-managed standby)
is untested on most of them, and mostly undocumented; pgnudge makes no claim
either way. The one confirmed data point is Azure Flexible Server, which
blocks it (see below). If you confirm it works, or that a platform blocks it,
open an issue and this table gets updated.
The WalFeed column reflects each vendor's own documentation (linked below).
pgnudge has not been integration-tested against any of these services;
verify against your plan and region, and let pgnudge doctor confirm the live
handshake.
| Platform | WalFeed (documented) |
RawFeed |
Enable wal_level = logical |
|---|---|---|---|
| AWS RDS PostgreSQL | yes | untested | rds.logical_replication=1, grant rds_replication |
| AWS Aurora PostgreSQL | yes | untested | rds.logical_replication=1 (cluster parameter group) |
| Google Cloud SQL | yes | untested | flag cloudsql.logical_decoding=on, user WITH REPLICATION |
| Azure Flexible Server | yes | no | wal_level=logical, ALTER ROLE ... WITH REPLICATION |
| Supabase | yes* | untested | role WITH REPLICATION; direct connection only |
| Neon | yes* | untested | enabling logical repl flips wal_level project-wide |
\* Supabase and Neon require a direct connection, not their pooler
(Supavisor / PgBouncer) - the same rule pgnudge already states for any pooler.
The RawFeed column is left untested: these vendors document logical
decoding as the external replication path and do not document an external
physical-streaming endpoint, but we have neither confirmed nor ruled one out.
Reports welcome.
Caveats worth a pre-flight pgnudge doctor:
- RDS / Aurora:
rds_replicationgrants logical-slot access but does not carry the rawREPLICATIONrole attribute; confirm the temporary-slotSTART_REPLICATIONpath. - Azure Flexible Server: external
START_REPLICATION PHYSICALis blocked (28000: no pg_hba.conf entry for replication connection), confirmed live viapgnudge doctor.RawFeedis unavailable; useWalFeed. Enablingwal_level=logicalneeds a server restart, and the login role needs theREPLICATIONattribute (grant it asazure_pg_admin). - Neon: enabling logical replication changes
wal_levelfor the whole project and cannot be undone. - Output plugin:
wal2jsonis common but not universal;test_decodingships with core PostgreSQL and is the zero-install fallback.
Sources (vendor docs): RDS logical replication, RDS/Aurora to self-managed, Aurora logical replication, Cloud SQL logical replication, Cloud SQL external server, Azure logical, Supabase external replication, Neon logical replication, Neon connection pooling.
Why not LISTEN/NOTIFY?
NOTIFY doesn't fire itself: making it track data changes means triggers, and
triggers are persistent catalog objects. Schema footprint, migration reviews,
cleanup jobs, drift. pgnudge's whole premise is refusing that trade. Logical
decoding gets the same wakeups straight from the WAL with zero objects. (LISTEN
is still great on the consuming side; see Fan-out.)
Fan-out
One WalFeed per process is the normal shape. For many consumers, run one
WalFeed in a small bridge daemon that republishes to a NOTIFY channel via
pg_notify, and let consumers attach with plain LISTEN (any driver; LISTEN is
session state, zero objects). One REPLICATION grant total, one decoding pass
total, and still zero persistent server objects: the bridge's temp slot dies
with the bridge.
Ops notes
status_interval(default 10 s) must stay under the server'swal_sender_timeout(default 60 s); the feed also answers reply-requested keepalives immediately.liveness_timeout(default 30 s, must exceedstatus_interval, enforced at construction;Nonedisables): each status report asks the server to answer with a keepalive, so a healthy connection always has inbound traffic. Silence longer than the timeout means a dead link (NAT drop, yanked VPN, hung walsender) and the feed aborts and reconnects instead of blocking forever.- While connected, each
WalFeedholds one replication slot and one WAL sender againstmax_replication_slots/max_wal_senders. Disconnected feeds hold nothing (that's the point), which also means an idle feed never retains WAL. - Managed platforms: enabling
wal_level=logicaltypically requires a restart (once); grantREPLICATIONto a dedicated role rather than widening an app role, since logical decoding sees the whole database's stream. On managed services, plan on that restart: whetherRawFeedcan serve as a no-restart alternative there is untested (external physical streaming is not known to be exposed; see Managed platforms). - A role with
REPLICATIONsees every table's changes through either transport regardless of its SELECT grants; table grants do not scope a change feed. Scope withtables=and treat the role as privileged. - Physical replication (
RawFeed) needs apg_hba.confentry for thereplicationpseudo-database (host replication <role> ...); the usualhost allrules do not match it. This is a self-hosted concern; whether managed platforms expose external physical streaming at all is untested (see Managed platforms). - Thundering herd: a database restart reconnects every feed at once, and every
consumer's
Resynchandler refetches at once. Reconnect timing is already jittered, but the refetch is your code. Add jitter there when many consumers share a database, or fan out through the bridge daemon so a single process refetches per change. - TLS:
ssl=Trueuses platform CA verification; pass anssl.SSLContextfor custom trust. SCRAM-SHA-256 is supported everywhere; cleartext auth only over TLS. pgnudge refuses to send a password on an unencrypted connection. - Logging: the
pgnudge.wallogger (stdliblogging, no handlers configured by the library) reports connect failures and stream errors at WARNING, successful (re)connects at INFO, and backoff timing at DEBUG. A feed that reconnects in a loop is visible, not silent. - Errors: every exception pgnudge raises inherits
PgnudgeError, soexcept PgnudgeErrorcatches them all.ConfigError(bad constructor argument) also inheritsValueError, so an existingexcept ValueErrorkeeps working. Stream and connection failures are internal lifecycle: the supervisor catches them, backs off, and reconnects with aResync; they do not surface on the iterator.
Tested how
The suite spins up real PostgreSQL via testcontainers (nothing to install beyond
Docker) and proves the claims live: no backfill of pre-connect writes,
client-side coalescing (50-row txn -> one Event, count=50), reconnect gets a
fresh slot with the old one auto-dropped, TLS + SCRAM over an encrypted stream,
and the flagship proof: hard socket abort with no protocol goodbye leaves
pg_replication_slots empty.
RawFeed gets its own proofs: pg_replication_slots stays empty while
streaming, an open transaction never nudges until COMMIT and a rollback never
nudges at all, VACUUM and CHECKPOINT stay silent, writes in other databases stay
silent, and an end-to-end run on an untouched wal_level=replica container. The
decoder itself is checked against an oracle: the same live WAL range through our
client-side walker and through pg_waldump must produce the identical change
sequence, on every PostgreSQL major in CI.
uv sync && uv run pytest
Non-goals
- Not a queue. No durability, no competing consumers, no retries. If a message must be processed, use a job queue (e.g. pgqueuer). pgnudge is its broadcast-shaped sibling: pgqueuer moves work, pgnudge moves wakefulness.
- Not CDC. No row images, no before/after, no replay. Refetch.
- Not a driver. The protocol client implements exactly what a logical-decoding consumer needs: startup, auth, simple query, CopyBoth.
Roadmap
- Native
pgoutputparsing would drop the wal2json server-plugin requirement, but pgoutput only decodes through a publication, and a publication is a persistent catalog object, in direct tension with the nothing-outlives-the-connection guarantee. Conditional at best: viable only if a pre-existing, application-owned publication counts as configuration rather than footprint. - Opt-in
schema.table:pkpayloads for sharper client-side routing. - The bridge daemon as a first-class artifact: same feed contract, one slot fanned out over NOTIFY; a native (Zig) implementation is the intended long-term core.
MIT licensed.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pgnudge-1.3.0.tar.gz.
File metadata
- Download URL: pgnudge-1.3.0.tar.gz
- Upload date:
- Size: 151.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9eec25823100a3ea2d49a5c349999983c1b6ce40d6e81a4c6dc11fb8e97eae95
|
|
| MD5 |
20800ce29ff73e8d73eba34f6cf4ec20
|
|
| BLAKE2b-256 |
8dcca2a00481e7a19b42a2566969908d745cb98b939acb63a9df350c71bf94e6
|
File details
Details for the file pgnudge-1.3.0-py3-none-any.whl.
File metadata
- Download URL: pgnudge-1.3.0-py3-none-any.whl
- Upload date:
- Size: 42.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a8eed72bd721b773cbaa1d6e871a87c5a6705f0ddc8546fe0fb54f8fe7985509
|
|
| MD5 |
54c9213f2b18e262963e46341e954b04
|
|
| BLAKE2b-256 |
6f6a069c650e9d10c20650c26279d2d4f9a0ddf640d526ce2f3b6490767fb32a
|