Patroni: A Template for PostgreSQL HA with ZooKeeper, etcd or Consul
You can find a searchable version of this documentation at patroni.readthedocs.io.
Patroni is a Python template for building PostgreSQL high availability (HA) clusters. It supports several distributed configuration stores, including ZooKeeper, etcd, Consul, and Kubernetes.
Supported PostgreSQL versions: 9.3 to 18.
Note to Citus users: Since version 3.0, Patroni integrates with the Citus extension. See the Citus support page for details.
Note to Kubernetes users: Patroni runs natively on Kubernetes. See the Kubernetes guide for more information.
How Patroni Works
Patroni (formerly known as Zalando’s Patroni) started as a fork of Governor and adds support for modern HA patterns.
For additional background info, see:
Development Status
Patroni is actively developed and welcomes contributions.
Community
Connect with the Patroni community on GitHub or Slack:
Requirements and Installation
Pre-requirements for macOS
Install the requirements on macOS with Homebrew:
brew install postgresql etcd haproxy libyaml python
Psycopg choices
Patroni requires a PostgreSQL Python driver. Recent versions of psycopg2 no longer install a binary package by default, which means building from source may require a C compiler and development libraries.
Options:
Install using the package manager from your Linux distribution:
sudo apt-get install python3-psycopg2 sudo yum install python3-psycopg2
Install one of the supported Python packages with pip:
psycopg
psycopg2
psycopg2-binary
Installing with pip
Install Patroni with optional dependency groups:
pip install patroni[dependencies]
Available dependency extras:
etcd or etcd3: python-etcd for Etcd as DCS
consul: py-consul for Consul as DCS
zookeeper: kazoo for ZooKeeper as DCS
exhibitor: kazoo for Exhibitor as DCS
kubernetes: kubernetes for Kubernetes as DCS
raft: pysyncobj for the python Raft DCS
aws: boto3 for AWS callbacks
systemd: systemd-python for sd_notify integration
all: all of the above (except psycopg family)
psycopg3: psycopg[binary]>=3.0.0
psycopg2: psycopg2>=2.5.4
psycopg2-binary: psycopg2-binary
For example:
pip install patroni[psycopg3,etcd3,aws]
Note: external tools used by bootstrap or replica creation scripts (for example WAL-G) must be installed separately.
Running and Configuring
A minimal cluster can be started from different terminals:
> etcd --data-dir=data/etcd --enable-v2=true > ./patroni.py postgres0.yml > ./patroni.py postgres1.yml
Then verify cluster behavior and experiment with the YAML configuration files.
Add more postgres*.yml files to scale the cluster.
Memory issue on Python 3.11+
If you run Patroni on a system with strict memory limits, for example with vm.overcommit_memory=2 (recommended for PostgreSQL), and use Python 3.11 or newer, you may observe unexpected behavior:
Patroni appears healthy
PostgreSQL continues to run
Patroni REST API becomes unresponsive
the operating system reports that Patroni is listening on the REST API port
Patroni logs look normal; however, following messages may appear once: Exception ignored in thread started by: <object repr() failed>, MemoryError
kernel logs may contain messages such as not enough memory for the allocation
This is caused by a Python 3.11+ issue. Under strict memory conditions, starting a new thread may hang indefinitely when there is not enough free memory.
Recommended solution
Recent Patroni releases (4.1.1+, 4.0.8+) reduce the impact of this issue by starting all required threads early in startup before memory pressure builds.
Additional recommendations (Linux, glibc)
When running with vm.overcommit_memory=2 (recommended for PostgreSQL), we also recommend starting Patroni with the following environment variables configured:
MALLOC_ARENA_MAX=1 - reduces the amount of virtual memory allocated by glibc for multi-threaded applications
PG_MALLOC_ARENA_MAX= - resets the value of MALLOC_ARENA_MAX for PostgreSQL processes started by Patroni.
In addition, you may tune the following Patroni configuration parameters:
thread_stack_size - stack size used for threads started by Patroni. Lowering this value reduces memory usage of the Patroni process. The default value set by Patroni is 512kB. Increase thread_stack_size if Patroni experience stack-related crashes; otherwise the default value is sufficient.
thread_pool_size - size of the thread pool used by Patroni for asynchronous tasks and REST API communication with other members during leader race or failsafe checks. The default value is 5, which is sufficient for three-node clusters.
restapi.thread_pool_size - size of the thread pool used to process REST API requests. The default value is 5, allowing up to five parallel REST API requests. Note that requests involving SQL queries are effectively serialized because a single database connection is used, so increasing this value typically provides no benefit.
HAProxy support
Patroni includes an HAProxy configuration for a single application endpoint. Start it with:
> haproxy -f haproxy.cfg
Then connect with:
> psql --host 127.0.0.1 --port 5000 postgres
Configuration References
YAML configuration
For complete YAML options, see docs/dynamic_configuration.rst and the example file postgres0.yml.
Environment configuration
For environment variable configuration, see docs/ENVIRONMENT.rst.
Replication choices
Patroni uses PostgreSQL streaming replication. It supports:
asynchronous replication with maximum_lag_on_failover
synchronous replication for stronger durability guarantees
See the replication modes documentation for details.
Application connections
Applications should connect with a non-superuser. Using a superuser can consume reserved connections for Patroni and cause undesirable behavior if the leader becomes unavailable.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file patroni-4.1.5.tar.gz.
File metadata
- Download URL: patroni-4.1.5.tar.gz
- Upload date:
- Size: 457.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9233df8cef01ac62e92cf6e564afe80ada284d87d2b4393874d807dde753cd05
|
|
| MD5 |
913d192abc3f225f158665728c6fa751
|
|
| BLAKE2b-256 |
63658e1ade8809da7a66971c20192c6ab0c9ecd6035952aea7380474e06def9e
|
Provenance
The following attestation bundles were made for patroni-4.1.5.tar.gz:
Publisher:
release.yaml on patroni/patroni
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
patroni-4.1.5.tar.gz -
Subject digest:
9233df8cef01ac62e92cf6e564afe80ada284d87d2b4393874d807dde753cd05 - Sigstore transparency entry: 2438033289
- Sigstore integration time:
-
Permalink:
patroni/patroni@235fdb3990513c7852da8dc51e9cb92b6d0e4907 -
Branch / Tag:
refs/tags/v4.1.5 - Owner: https://github.com/patroni
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yaml@235fdb3990513c7852da8dc51e9cb92b6d0e4907 -
Trigger Event:
release
-
Statement type:
File details
Details for the file patroni-4.1.5-py3-none-any.whl.
File metadata
- Download URL: patroni-4.1.5-py3-none-any.whl
- Upload date:
- Size: 378.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7aad8c1539e6c51c269a8005169dd54ddc7668437b7900fa6099d7894e224c84
|
|
| MD5 |
98c1f734dc67bfae7336d291c66a92b2
|
|
| BLAKE2b-256 |
63cb0ba365e95d648a0592de0fb1a0168cff26a3cf57930d74d88b188b5f9462
|
Provenance
The following attestation bundles were made for patroni-4.1.5-py3-none-any.whl:
Publisher:
release.yaml on patroni/patroni
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
patroni-4.1.5-py3-none-any.whl -
Subject digest:
7aad8c1539e6c51c269a8005169dd54ddc7668437b7900fa6099d7894e224c84 - Sigstore transparency entry: 2438033309
- Sigstore integration time:
-
Permalink:
patroni/patroni@235fdb3990513c7852da8dc51e9cb92b6d0e4907 -
Branch / Tag:
refs/tags/v4.1.5 - Owner: https://github.com/patroni
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yaml@235fdb3990513c7852da8dc51e9cb92b6d0e4907 -
Trigger Event:
release
-
Statement type: