Skip to main content

A commandline tool to anonymize PostgreSQL databases for DSGVO/GDPR purposes.

It uses a YAML file to define which tables and fields should be anonymized and provides various methods of anonymization. The tool requires a direct PostgreSQL connection to perform the anonymization.

PyPI - Python Version license pypi Download count build codecov pganonymize

https://raw.githubusercontent.com/rheinwerk-verlag/pganonymize/main/docs/_static/demo.gif

Features

  • Intentionally compatible with Python 2.7 (for old, productive platforms)

  • Anonymize PostgreSQL tables on data level entry with various providers (some examples in the table below)

  • Exclude data for anonymization depending on regular expressions or SQL WHERE clauses

  • Truncate entire tables for unwanted data

Field

Value

Provider

Output

first_name

John

choice

(Bob|Larry|Lisa)

title

Dr.

clear

street

Irving St

faker.street_name

Miller Station

password

dsf82hFxcM

mask

XXXXXXXXXX

credit_card

1234-567-890

partial_mask

1??????????0

email

jane.doe@example.com

md5

0cba00ca3da1b283a57287bcceb17e35

email

jane.doe@example.com

faker.unique.email

alex7@sample.com

phone_num

65923473

md5 as_number: True

3948293448

ip

157.50.1.20

set

127.0.0.1

uuid_col

00010203-0405-……

uuid4

f7c1bd87-4d….

  • Note: faker.unique.[provider] only supported on Python 3.6+ (Faker library min. supported python version)

  • Note: uuid4 - only for (native uuid4) columns

See the documentation for a more detailed description of the provided anonymization methods.

Installation

The default installation method is to use pip:

$ pip install pganonymize

Usage

usage: pganonymize [-h] [-v] [-l] [--schema SCHEMA] [--dbname DBNAME]
               [--user USER] [--password PASSWORD] [--host HOST]
               [--port PORT] [--dry-run] [--dump-file DUMP_FILE]

Anonymize data of a PostgreSQL database

optional arguments:
-h, --help            show this help message and exit
-v, --verbose         Increase verbosity
-l, --list-providers  Show a list of all available providers
--schema SCHEMA       A YAML schema file that contains the anonymization
                        rules
--dbname DBNAME       Name of the database
--user USER           Name of the database user
--password PASSWORD   Password for the database user
--host HOST           Database hostname
--port PORT           Port of the database
--dry-run             Don't commit changes made on the database
--dump-file DUMP_FILE
                      Create a database dump file with the given name
--dump-options DUMP_OPTIONS
                      Options to pass to the pg_dump command
--init-sql INIT_SQL   SQL to run before starting anonymization
--parallel            Data anonymization is done in parallel

Despite the database connection values, you will have to define a YAML schema file, that includes all anonymization rules for that database. Take a look at the schema documentation or the YAML sample schema.

Example calls:

$ pganonymize --schema=myschema.yml \
    --dbname=test_database \
    --user=username \
    --password=mysecret \
    --host=db.host.example.com \
    -v

$ pganonymize --schema=myschema.yml \
    --dbname=test_database \
    --user=username \
    --password=mysecret \
    --host=db.host.example.com \
    --init-sql "set search_path to non_public_search_path; set work_mem to '1GB';" \
    -v

Database dump

With the --dump-file argument it is possible to create a dump file after anonymizing the database. Please note, that the pg_dump command from the postgresql-client-common library is necessary to create the dump file for the database, e.g. under Linux:

$ sudo apt-get install postgresql-client-common

Example call:

$ pganonymize --schema=myschema.yml \
    --dbname=test_database \
    --user=username \
    --password=mysecret \
    --host=db.host.example.com \
    --dump-file=/tmp/dump.gz \
    -v

So that the password for dumping does not have to be entered manually, it can also be entered as an environment var PGPASSWORD:

$ PGPASSWORD=password pganonymize --schema=myschema.yml \
    --dbname=test_database \
    --user=username \
    --password=mysecret \
    --host=db.host.example.com \
    --dump-file=/tmp/dump.gz \
    -v

Docker

If you want to run the anonymizer within a Docker container you first have to build the image:

$ docker build -t pganonymize .

After that you can pass a schema file to the container, using Docker volumes, and call the anonymizer:

$ docker run \
    -v <path to your schema>:/schema.yml \
    -it pganonymize \
    /usr/local/bin/pganonymize \
    --schema=/schema.yml \
    --dbname=<database> \
    --user=<user> \
    --password=<password> \
    --host=<host> \
    -v

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pganonymize-0.13.0.tar.gz (24.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pganonymize-0.13.0-py2.py3-none-any.whl (16.1 kB view details)

Uploaded Python 2Python 3

File details

Details for the file pganonymize-0.13.0.tar.gz.

File metadata

  • Download URL: pganonymize-0.13.0.tar.gz
  • Upload date:
  • Size: 24.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pganonymize-0.13.0.tar.gz
Algorithm Hash digest
SHA256 9dc1d7b6fca93de2c3b189198b0344234f0c0ee1385ec7bd48e97d4ef05af5cf
MD5 62d4b78ea86a347b9cc5fdcb5d633759
BLAKE2b-256 047b3f72694cf96462a37bf5682e69fb8a9f431f64dbdca21c6eb293e4bb540b

See more details on using hashes here.

Provenance

The following attestation bundles were made for pganonymize-0.13.0.tar.gz:

Publisher: publish.yml on rheinwerk-verlag/pganonymize

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pganonymize-0.13.0-py2.py3-none-any.whl.

File metadata

  • Download URL: pganonymize-0.13.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 16.1 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pganonymize-0.13.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 0f81d2ad4cc18bd5cccfaed327716071ccaa34a6319bb62c5bc3144f14df00b8
MD5 e80599dd8356e8f8f66675e33571f43d
BLAKE2b-256 a228bcad30cd42e74c6c9b8b96ac8f40a4a860564077adf14e8704a2fe848a99

See more details on using hashes here.

Provenance

The following attestation bundles were made for pganonymize-0.13.0-py2.py3-none-any.whl:

Publisher: publish.yml on rheinwerk-verlag/pganonymize

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.13.0 This release

2 files

0.12.0

2 files

0.11.0

2 files

0.10.0

2 files

0.9.0

2 files

0.8.0

2 files

0.7.0

2 files

0.6.1

2 files

0.6.0

2 files

0.5.0

3 files

0.4.1

2 files

0.4.0

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.4

1 file

0.2.3

1 file

0.2.2

1 file

0.2.1

1 file

0.2.0

1 file

0.1.1

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page