Skip to main content

Tests Status Coverage Status

Patroni: A Template for PostgreSQL HA with ZooKeeper, etcd or Consul

You can find a searchable version of this documentation at patroni.readthedocs.io.

Patroni is a Python template for building PostgreSQL high availability (HA) clusters. It supports several distributed configuration stores, including ZooKeeper, etcd, Consul, and Kubernetes.

Supported PostgreSQL versions: 9.3 to 18.

Note to Citus users: Since version 3.0, Patroni integrates with the Citus extension. See the Citus support page for details.

Note to Kubernetes users: Patroni runs natively on Kubernetes. See the Kubernetes guide for more information.

How Patroni Works

Patroni (formerly known as Zalando’s Patroni) started as a fork of Governor and adds support for modern HA patterns.

For additional background info, see:

Development Status

Patroni is actively developed and welcomes contributions.

Community

Connect with the Patroni community on GitHub or Slack:

Requirements and Installation

Pre-requirements for macOS

Install the requirements on macOS with Homebrew:

brew install postgresql etcd haproxy libyaml python

Psycopg choices

Patroni requires a PostgreSQL Python driver. Recent versions of psycopg2 no longer install a binary package by default, which means building from source may require a C compiler and development libraries.

Options:

  1. Install using the package manager from your Linux distribution:

sudo apt-get install python3-psycopg2
sudo yum install python3-psycopg2
  1. Install one of the supported Python packages with pip:

  • psycopg

  • psycopg2

  • psycopg2-binary

Installing with pip

Install Patroni with optional dependency groups:

pip install patroni[dependencies]

Available dependency extras:

  • etcd or etcd3: python-etcd for Etcd as DCS

  • consul: py-consul for Consul as DCS

  • zookeeper: kazoo for ZooKeeper as DCS

  • exhibitor: kazoo for Exhibitor as DCS

  • kubernetes: kubernetes for Kubernetes as DCS

  • raft: pysyncobj for the python Raft DCS

  • aws: boto3 for AWS callbacks

  • systemd: systemd-python for sd_notify integration

  • all: all of the above (except psycopg family)

  • psycopg3: psycopg[binary]>=3.0.0

  • psycopg2: psycopg2>=2.5.4

  • psycopg2-binary: psycopg2-binary

For example:

pip install patroni[psycopg3,etcd3,aws]

Note: external tools used by bootstrap or replica creation scripts (for example WAL-G) must be installed separately.

Running and Configuring

A minimal cluster can be started from different terminals:

> etcd --data-dir=data/etcd --enable-v2=true
> ./patroni.py postgres0.yml
> ./patroni.py postgres1.yml

Then verify cluster behavior and experiment with the YAML configuration files.

Add more postgres*.yml files to scale the cluster.

Memory issue on Python 3.11+

If you run Patroni on a system with strict memory limits, for example with vm.overcommit_memory=2 (recommended for PostgreSQL), and use Python 3.11 or newer, you may observe unexpected behavior:

  • Patroni appears healthy

  • PostgreSQL continues to run

  • Patroni REST API becomes unresponsive

  • the operating system reports that Patroni is listening on the REST API port

  • Patroni logs look normal; however, following messages may appear once: Exception ignored in thread started by: <object repr() failed>, MemoryError

  • kernel logs may contain messages such as not enough memory for the allocation

This is caused by a Python 3.11+ issue. Under strict memory conditions, starting a new thread may hang indefinitely when there is not enough free memory.

Additional recommendations (Linux, glibc)

When running with vm.overcommit_memory=2 (recommended for PostgreSQL), we also recommend starting Patroni with the following environment variables configured:

  • MALLOC_ARENA_MAX=1 - reduces the amount of virtual memory allocated by glibc for multi-threaded applications

  • PG_MALLOC_ARENA_MAX= - resets the value of MALLOC_ARENA_MAX for PostgreSQL processes started by Patroni.

In addition, you may tune the following Patroni configuration parameters:

  • thread_stack_size - stack size used for threads started by Patroni. Lowering this value reduces memory usage of the Patroni process. The default value set by Patroni is 512kB. Increase thread_stack_size if Patroni experience stack-related crashes; otherwise the default value is sufficient.

  • thread_pool_size - size of the thread pool used by Patroni for asynchronous tasks and REST API communication with other members during leader race or failsafe checks. The default value is 5, which is sufficient for three-node clusters.

  • restapi.thread_pool_size - size of the thread pool used to process REST API requests. The default value is 5, allowing up to five parallel REST API requests. Note that requests involving SQL queries are effectively serialized because a single database connection is used, so increasing this value typically provides no benefit.

HAProxy support

Patroni includes an HAProxy configuration for a single application endpoint. Start it with:

> haproxy -f haproxy.cfg

Then connect with:

> psql --host 127.0.0.1 --port 5000 postgres

Configuration References

YAML configuration

For complete YAML options, see docs/dynamic_configuration.rst and the example file postgres0.yml.

Environment configuration

For environment variable configuration, see docs/ENVIRONMENT.rst.

Replication choices

Patroni uses PostgreSQL streaming replication. It supports:

  • asynchronous replication with maximum_lag_on_failover

  • synchronous replication for stronger durability guarantees

See the replication modes documentation for details.

Application connections

Applications should connect with a non-superuser. Using a superuser can consume reserved connections for Patroni and cause undesirable behavior if the leader becomes unavailable.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

patroni-4.1.5.tar.gz (457.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

patroni-4.1.5-py3-none-any.whl (378.0 kB view details)

Uploaded Python 3

File details

Details for the file patroni-4.1.5.tar.gz.

File metadata

  • Download URL: patroni-4.1.5.tar.gz
  • Upload date:
  • Size: 457.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for patroni-4.1.5.tar.gz
Algorithm Hash digest
SHA256 9233df8cef01ac62e92cf6e564afe80ada284d87d2b4393874d807dde753cd05
MD5 913d192abc3f225f158665728c6fa751
BLAKE2b-256 63658e1ade8809da7a66971c20192c6ab0c9ecd6035952aea7380474e06def9e

See more details on using hashes here.

Provenance

The following attestation bundles were made for patroni-4.1.5.tar.gz:

Publisher: release.yaml on patroni/patroni

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file patroni-4.1.5-py3-none-any.whl.

File metadata

  • Download URL: patroni-4.1.5-py3-none-any.whl
  • Upload date:
  • Size: 378.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for patroni-4.1.5-py3-none-any.whl
Algorithm Hash digest
SHA256 7aad8c1539e6c51c269a8005169dd54ddc7668437b7900fa6099d7894e224c84
MD5 98c1f734dc67bfae7336d291c66a92b2
BLAKE2b-256 63cb0ba365e95d648a0592de0fb1a0168cff26a3cf57930d74d88b188b5f9462

See more details on using hashes here.

Provenance

The following attestation bundles were made for patroni-4.1.5-py3-none-any.whl:

Publisher: release.yaml on patroni/patroni

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

4.1.5 This release

2 files

4.1.4

2 files

4.1.3

2 files

4.1.2

2 files

4.1.1

2 files

4.1.0

2 files

4.0.11

2 files

4.0.10

2 files

4.0.9

2 files

4.0.8

2 files

4.0.7

2 files

4.0.6

2 files

4.0.5

2 files

4.0.4

2 files

4.0.3

2 files

4.0.2

2 files

4.0.1

2 files

4.0.0

2 files

3.3.11

2 files

3.3.10

2 files

3.3.9

2 files

3.3.8

2 files

3.3.7

2 files

3.3.6

2 files

3.3.5

2 files

3.3.4

2 files

3.3.3

2 files

3.3.2

2 files

3.3.1

2 files

3.3.0

2 files

3.2.2

2 files

3.2.1

2 files

3.2.0

2 files

3.1.2

2 files

3.1.1

2 files

3.1.0

2 files

3.0.4

2 files

3.0.3

2 files

3.0.2

2 files

3.0.1

2 files

3.0.0

2 files

2.1.7

2 files

2.1.6

2 files

2.1.5

2 files

2.1.4

2 files

2.1.3

2 files

2.1.2

2 files

2.1.1

2 files

2.1.0

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

1.6.5

2 files

1.6.4

2 files

1.6.3

2 files

1.6.1

2 files

1.6.0

2 files

1.5.6

2 files

1.5.5

2 files

1.5.4

2 files

1.5.3

2 files

1.5.2

2 files

1.5.1

2 files

1.5.0

2 files

1.4.6

2 files

1.4.4

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4

2 files

1.3.6

2 files

1.3.5

2 files

1.3.4

2 files

1.3.3

2 files

1.3.2

2 files

1.3.1

2 files

1.3

2 files

1.2.5

2 files

1.2.4

2 files

1.2.3

2 files

1.2.2

2 files

1.2.1

2 files

1.2

2 files

1.1

2 files

1.0

2 files

0.90

2 files

0.80

2 files

0.76

2 files

0.75

2 files

0.7

2 files

0.6

0.2

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page