This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

A clone of django.db.backends.postgresql that supports database defaults. For use with Django 1.9.x.

Motivation - better support database migrations with rolling web server updates

The Django migrations (new in Django 1.7) do not insert default values into the database. I would like database defaults to allow me to update the database with new migrations while it is still running an older version of a web app. For example, the majority of my updates follow this general pattern:

  1. remove one machine from my production load balancer
  2. run the latest database migrations for the database used by all production web servers
  3. update the code on my machine that is out of the load balancer and verify things work directly on that server
  4. udpate the remaining web servers with the latest code

This creates a delay between the database update and the newest code, which could lead to failed database inserts during that period if a column is not null and its default is not set in the database.

Some people suggest creating an additional data migration script to add the defaults by hand, which would help solve the problem, but (a) this is easy to miss and (b) I’m guessing this would create a brief window of time where the column exists without a default and could still lead to a failed insert.

Furthermore, some of the [existing documentation](https://docs.djangoproject.com/en/1.7/topics/migrations/#postgresql) is vague and doesn’t make it clear that database defaults are not included in migrations.

Solution

For the django.db.backends.postgresql module, I have noticed that it actually adds the defaults in the db when it creates columns, but then it removes them in a separate SQL statement. As a result, I have made a subclass of the module that overrides the one string that drops the default and replaces it with a no-op. (This is admittedly a hack, but appears to be the most minimal approach that will survive more updates than copy-pasting large sections of the original class.)

For example postgresql might generate SQL for a migration as such:

$ python3 manage.py sqlmigrate app 0010
BEGIN;
ALTER TABLE "my_table" ALTER COLUMN "some_column" SET DEFAULT false;
ALTER TABLE "my_table" ALTER COLUMN "some_column" DROP DEFAULT;
COMMIT;

With this change, the same migration script turns the 2nd command into a no-op:

$ python3 manage.py sqlmigrate app 0010
BEGIN;
ALTER TABLE "my_table" ALTER COLUMN "some_column" SET DEFAULT false;
ALTER TABLE "my_table" DROP COLUMN IF EXISTS skip_django_drop_default_feature RESTRICT;
COMMIT;

Furthermore, there’s no need to worry about migrations that intend to do DROP DEFAULT, because the django migrations code already doesn’t do anything in SQL for such a change, it would actually look like this:

$ python3 manage.py sqlmigrate app 0011
BEGIN;
--
-- Alter field some_column on my_table
--

COMMIT;

Discussion

I have been using this in production on ringly.com for over a year and safely avoided running into the issues such as a new char field with null=False and default=” from causing insert errors while some app servers are on new code and others are still on the old code.

I would love to hear ideas that people have about this approach—it seems like it may have been controversial in the past? In my limited use with it, it appears to solve my needs, but I haven’t seen what happens in more complex scenarios such as using a callable for a default (IMO it’s reasonable not to support that specific scenario).

Some discussion on django-developers:

Another thought, it maybe adding more flags to migrations that allows deeper customization, but more flags and feature bloat are definitely dangerous things with any framework.

Regardless of what I end up using, I have setup a more rigorous workflow for review migrations before developers push them into our release branch.

Release History

Release History

0.1.1

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
django-postgres-dbdefaults-0.1.1.tar.gz (4.9 kB) Copy SHA256 Checksum SHA256 Source Apr 27, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting