Skip to main content

Consistent partial database dump utility

Project description


Build Status Coverage Status Documentation Status Latest PyPI version

XDump is an utility to make partial consistent dump and load it into the database.

The idea is to provide an ability to specify what to include in the dump via SQL queries.


XDump can be obtained with pip:

$ pip install xdump

Usage example

Making a dump (on production replica for example):

>>> from xdump.postgresql import PostgreSQLBackend
>>> backend = PostgreSQLBackend(dbname='app_db', user='prod', password='pass', host='', port='5432')
>>> backend.dump(
    partial_tables={'employees': 'SELECT * FROM employees ORDER BY id DESC LIMIT 2'}

Load a dump on you local machine:

>>> backend = PostgreSQLBackend(dbname='app_db', user='local', password='pass', host='', port='5432')
>>> backend.load('/path/to/')

Dump is compressed by default. Compression level could be changed with passing compression argument to dump method. Valid options are zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED, zipfile.ZIP_BZIP2 and zipfile.ZIP_LZMA.

Verbosity of the output could be customized via verbosity (with values 0, 1 or 2) argument of a backend class.

There are two options to control the content of the dump:

  • dump_schema - controls if the schema should be included

  • dump_data - controls if the data should be included

RDBMS support

At the moment only the following are supported:

  • PostgreSQL

  • SQLite >= 3.8.3

Django support

Add xdump.extra.django to your INSTALLED_APPS settings:


Add XDUMP to your project settings file. It should contain minimum 2 entries:

  • FULL_TABLES - a list of tables, that should be fully dumped.

  • PARTIAL_TABLES - a dictionary with table_name: select SQL

    'FULL_TABLES': ['groups'],
    'PARTIAL_TABLES': {'employees': 'SELECT * FROM employees WHERE id > 100'}

Optionally you could use a custom backend:

    'BACKEND': 'importable.string',

Run xdump command:

$ ./ xdump

Run xload command:

$ ./ xload

Possible options to both commands:

  • alias - allows you to choose database config from DATABASES, that is used during the execution;

  • backend - importable string, that leads to custom dump backend class.

  • dump_schema - controls if the schema should be included

  • dump_data - controls if the data should be included

The following make command could be useful to get a configured dump from production to your local machine:

    ssh -t $(TARGET) "DJANGO_SETTINGS_MODULE=settings.production /path/to/ xdump /tmp/"
    scp $(TARGET):/tmp/ ./
    ssh -t $(TARGET) "rm /tmp/"
    DJANGO_SETTINGS_MODULE=settings.local $(PYTHON) xload ./

And usage is:

$ make sync-production PYTHON=/path/to/python/in/venv

Python support

XDump supports Python 2.7, 3.4 - 3.7 and PyPy 2 & 3.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

xdump-0.5.0.tar.gz (13.6 kB view hashes)

Uploaded Source

Built Distribution

xdump-0.5.0-py2.py3-none-any.whl (14.6 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page