Skip to main content

Run a few sanity checks on your logical PostgreSQL replica to make sure its the same as the primary.

Project description

pg-replica-auditor

Features

Assumptions

These tests assume that id and updated_at columns exist and have indexes for efficient querying and that the table exists on both databases.

Row comparison

Runs row comparisons between primary and replica using two methods:

  1. select 8128 rows (or number of rows given to --rows) at random between MIN(id) and MAX(id)
  2. select the last 1000 rows.

Replica lag

Checks for "replica lag" by comparing MAX(updated_at) on the given table on both databases.

MinMax

Checks that the minimum id and the maximum id match on both replica and primary. These can drift a little because of replica lag.

Bulk 1000 Sum

Take the sum of the id column in chunks of 1000 and compare it between databases. This assumes that retrieving rows in bulk is easier than at random and runs faster than the row comparison and can scan more rows.

Requirements

  1. Python 3
  2. Postgres development files (required by psycopg2). On Mac OS, use brew install postgresql. On Ubuntu, install libpq-dev.

Installation

Development

Using virtualenv, pip install -r requirements.txt

Production

Using Pypi, pip install pg-replica-auditor.

Usage

This script requires three arguments:

  1. --primary, any acceptable Postgres connection string (incl. DSN),
  2. --replica, same as --primary but for the replica database,

Optional arguments:

  1. --exclude-tables, excludes the comma-separated tables from the scan,
  2. --table, only scans this table,
  3. --debug, will print debugging information,
  4. --rows, will scan this many rows in the row comparisons check.

Example:

$ pgreplicaauditor --primary=postgres://primary-db.amazonaws.com:5432/my_db --replica=postgres://replica-db.amazonaws.com:5432/my_db --table=immutable_items

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pg-replica-auditor, version 0.3.1
Filename, size File type Python version Upload date Hashes
Filename, size pg_replica_auditor-0.3.1-py3-none-any.whl (6.3 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size pg-replica-auditor-0.3.1.tar.gz (4.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page