Skip to main content
Build status

This tool allows to extract data from a PostgreSQL database with greater flexibility that tools like pg_dump allow.

pg_seldump reads one of more dump definitions from YAML files and selects what tables or other database objects to save. It is possible to extract only certain columns of the tables, only certain records, or to replace certain values with a different expression, for instance to anonymize data.

The output of the program is a text file which can be used by psql to restore data into a database with a complete schema but with no data (or at least no conflicting data), e.g. using:

$ pg_seldump --dsn="dbname=sourcedb" datadump.yaml > dump.sql
...
$ psql -1X --set ON_ERROR_STOP=1 -f dump.sql "dbname=targetdb"

Program usage

Usage:

pg_seldump [-h] [--version] [--dsn DSN] [--outfile OUTFILE] [--test]
           [-q | -v]
           config [config ...]

Create a selective dump of a PostgreSQL database.

positional arguments:
  config                yaml file describing the data to dump

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --dsn DSN             database connection string [default: '']
  --outfile OUTFILE, -o OUTFILE
                        the file where to save the dump [default: stdout]
  --test                test the configuration to verify it works as expected
  -q, --quiet           talk less
  -v, --verbose         talk more

The config files must be YAML files containing a db_objects list of entries. Each entry may have:

Selectors (all the specified ones must match):

  • name: name of the db object to dump

  • names: list of names or regex of db objects to dump

  • schema: schema name of the db object to dump

  • schemas: list of schema names or regexp to match schema names of the db object to dump

  • kind: kind of object to match. Can be:

    • table

    • sequence

    • paritioned table

    • materialized view

  • kinds: list of kind of objects to match (like for kind)

  • adjust_score: adjustment for the match score to break rules ties

Data modifiers:

  • action: what to do with the matched object:

    • dump: dump the object in the output (default)

    • skip: don’t dump the object

    • error: raise an error in case of match (useful to create strict descriptions where all the db objects must be mentioned explicitly)

  • no_columns: list of columns names to omit

  • filter: WHERE condition to include only a subset of the records in the dump

  • replace: mapping from column names to SQL expressions to replace values into the dump with somethings else

The objects in the database are matched to the rules in the config files. Every match will have a score according to how specific was the selector matched the object.

  • name or names list: 1000

  • names regexp: 500

  • schema or schemas list: 100

  • schemas regexp: 50

  • kind or kinds: 10

The rule with the highest score will apply. If two rules have exactly the same score the program will report an error: you can use adjust_score to break the tie.

Metadata

Release files for pg-seldump 0.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pg-seldump 0.3.1
File Size Uploaded
pg_seldump-0.3.1.tar.gz 28.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for pg-seldump 0.3.1
File Interpreter ABI Platform
pg_seldump-0.3.1-py3-none-any.whl Python 3 none any Details

Total release size: 57.0 kB

Release files / pg_seldump-0.3.1.tar.gz

Download URL pg_seldump-0.3.1.tar.gz
Size 28.9 kB
Tags Source
SHA-256 checksum
How to use checksums
bece9426a0109b65d52a567aa47c25778f4008f2be677335ada1dd9d5c1578f4
BLAKE2b-256 checksum
How to use checksums
2e865ea27ae08d0584f14e48f50538541876c8d4f572c07407af2cf067722de5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.12

Release files / pg_seldump-0.3.1-py3-none-any.whl

Download URL pg_seldump-0.3.1-py3-none-any.whl
Size 28.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
527db225abc0a86706fd47509b38fb2fbc334913aa8d7f871dfc10712edcc41d
BLAKE2b-256 checksum
How to use checksums
43d9a28abfd86d2c581c738ce4ebd8af8fc5cbd91a6f18d5afd8bda7c0aadf9d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.12

Release history Release notifications | RSS feed

0.3.3

2 release files

0.3.2

2 release files

This release

0.3.1 This release

2 release files

0.3

1 release file

0.2

1 release file

0.1

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page