Skip to main content

Extraction tool for data only django migrations

Project description

django-data-migration

Test dev branch Pypi

Developing and maintaining a django project over many years can start to become a constant fight against time consuming tasks including execution of a test suite, recreation of a local environment or setting up a project in a new environment.
Due to different flavors of deployment and/or different approaches within the same working environment migration files of long running django applications tent to be bloated and contain unnecessary code. Sometimes we even create migrations in the purpose of single-time usage to move, edit, duplicate or basically modify data.
Generally speaking, the idea behind it is clever.
With this approach you gained the option to trigger the execution of leaf migrations prior to starting your updated application code.
      Missing migration?
      /                \
yes, migrate        no, continue
      \                /
   restart app with new code
But on the other hand you create a new node within a already giant migration graph.
This is where django-data-migration comes in place. It is a drop-in replacement for regular migrations, without the need of a dedicated node in the migration tree.
It does that, by providing a “data-only” migration graph, that can optionally be maintained automatically in parallel with the existing migration graph, or executed independently, depending on your needs.

Installation

Install package:

pip install django-data-migrations
Configure package in Django settings:
INSTALLED_APPS = [
    # django apps
    'data_migration',
    # your apps
]

Configuration

The package is configurable using the
DATA_MIGRATION = {}

setting.

Currently supported attributes:

  • SQUASHABLE_APPS: a list of app(-label) names which allow squashing, you should only provide your own apps here

Usage

Extended management commands: - makemigrations - migrate - squashmigrations - data_migrate

makemigrations

# generate data migration file
./manage.py makemigrations --data-only [app_name]

# generate data migration file with readable name "name_change"
./manage.py makemigrations --data-only [app_name] name_change

# generate empty file
./manage.py makemigrations --data-only [app_name] --empty

# generate without fileheader
./manage.py makemigrations --data-only [app_name] --no-header

The makemigrations command generates a file [app_name]/data_migrations/[id]_[name].py with content like

class Node:
    name = '0001_first'
    dependencies = ()
    migration_dependencies = ('testapp.0001_initial', )
    routines = [
    ]

migrate

# apply data migration file
./manage.py migrate --data-only

# revert complete data migration state
./manage.py migrate --data-only zero

# revert partial data migration state
./manage.py migrate --data-only 0002_some_big_change

squashmigrations

App-wise squashing of data/regular migrations.
# regular squashing of test_app migrations 0001-0015
./manage.py squashmigrations test_app 0001 0015

# squash and replace test_app migrations 0001-0015 and extract data_migrations
./manage.py squashmigrations --extract-data-migrations test_app 0001 0015

data_migrate

Extended squashing. Allows squashing a single app, a list of apps, or all apps at once.
# squash and replace all migrations at once
./manage.py data_migrate --all

# squash and replace migrations app-wise
./manage.py data_migrate test_app

Development

To develop the package further set up a local environment using the provided ./dev-requirements.txt file.

To run the test suite and generate a coverage report run

coverage run -m pytest -v tests
coverage [html|report]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-data-migrations-0.1.5.tar.gz (15.4 kB view details)

Uploaded Source

Built Distribution

django_data_migrations-0.1.5-py3-none-any.whl (19.2 kB view details)

Uploaded Python 3

File details

Details for the file django-data-migrations-0.1.5.tar.gz.

File metadata

File hashes

Hashes for django-data-migrations-0.1.5.tar.gz
Algorithm Hash digest
SHA256 246cd037d2d19b83c6a2da7e5207eeeec8a139a0e70b6b9ba54bc9cdf38f0a8a
MD5 781aef7d6df00c128a29bddad8e27d87
BLAKE2b-256 1ca4e36210c3f4c4b87dd5811f8edf73abd194df7020c91cc1ebad4dddebdeed

See more details on using hashes here.

File details

Details for the file django_data_migrations-0.1.5-py3-none-any.whl.

File metadata

File hashes

Hashes for django_data_migrations-0.1.5-py3-none-any.whl
Algorithm Hash digest
SHA256 d97f18946ae69188155b7923c3396de88d2575dc17b99981746dbec8d52a5b18
MD5 fedcfbce567576a977124cbc0003ca4b
BLAKE2b-256 7094eb0dda3cae6955c083b9fb4219651d0eacea55937e5c94ea7d098f6ab7f0

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page