Django Migrations for raw SQL
Project description
django-migrate-sql-deux
Source Code: https://github.com/browniebroke/django-migrate-sql-deux
Django Migrations support for raw SQL.
[!NOTE] This package is a fork of the
django-migrate-sql
package, originally published by Bogdan Klichuk. This package appears unmaintained, so we decided to start a fork as we depended on it. Most of the code is from the original author.
About
This tool implements mechanism for managing changes to custom SQL entities (functions, types, indices, triggers) using built-in migration mechanism. Technically creates a sophistication layer on top of the RunSQL
Django operation.
What it does
- Makes maintaining your SQL functions, custom composite types, indices and triggers easier.
- Structures SQL into configuration of SQL items, that are identified by names and divided among apps, just like models.
- Automatically gathers and persists changes of your custom SQL into migrations using
makemigrations
. - Properly executes backwards/forwards keeping integrity of database.
- Create -> Drop -> Recreate approach for changes to items that do not support altering and require dropping and recreating.
- Dependencies system for SQL items, which solves the problem of updating items, that rely on others (for example custom types/functions that use other custom types), and require dropping all dependency tree previously with further recreation.
What it does not
- Does not parse SQL nor validate queries during
makemigrations
ormigrate
because is database-agnostic. For this same reason setting up proper dependencies is user's responsibility. - Does not create
ALTER
queries for items that support this, for exampleALTER TYPE
in PostgreSQL, because is database-agnostic. In case your tools allow rolling all the changes throughALTER
queries, you can consider not using this app or restructure migrations manually after creation by nesting generated operations into`state_operations
ofRunSQL
that doesALTER
. - (TODO)During
migrate
does not restore full state of items for analysis, thus does not notify about existing changes to schema that are not migrated nor does not recognize circular dependencies during migration execution.
Installation
Install from PyPi:
pip install django-migrate-sql-deux
Add django_migrate_sql
to INSTALLED_APPS
:
INSTALLED_APPS = [
# ...
'django_migrate_sql',
]
App defines a custom makemigrations
command, that inherits from Django's core one, so in order django_migrate_sql
app to kick in, put it before any other apps that redefine makemigrations
command too, following the official guide on how to override commands.
Integrations with other apps
If you want functionality from multiple makemigrations
commands provided by other apps, you may have to create your own command, inheriting from all the base commands. The core functionality is provided as a mixin DjangoMigrateSQLMixin
to make this easier. For instance, to integrate with django-linear-migrations
, you would write a makemigrations
command along these lines:
# app1/management/commands/makemigrations.py
from django_linear_migrations.management.commands.makemigrations import Command as LinearMigrationsMakeMigrationsCommand
from django_migrate_sql.management.commands.makemigrations import DjangoMigrateSQLMixin
class Command(DjangoMigrateSQLMixin, LinearMigrationsMakeMigrationsCommand):
pass
Again, you should make sure that the app where your custom command is implemented is placed above all the other apps.
Django 5.2+
If you're running Django 5.2 or newer, you should also do something similar for the migrate
command. Django 5.2 introduces a system check to make sure both commands use the same autodetector
class.
If you run another app providing a custom autodetector
class, you'll need to combine it with the autodetector
from this package.
# app1/management/commands/_autodetector.py
from django_migrate_sql.autodetector import MigrationAutodetector as MigrateSQLMigrationAutodetector
from other_app.autodetector import MigrationAutodetector as OtherAppMigrationAutodetector
class MigrationAutodetector(MigrateSQLMigrationAutodetector, OtherAppMigrationAutodetector):
pass
# app1/management/commands/makemigrations.py
# Assuming "other_app" provides a custom makemigrations command,
# this file is not needed if not
from other_app.management.commands.makemigrations import (
Command as MakeMigrationsCommand,
)
from ._autodetector import MigrationAutodetector
class Command(MakeMigrationsCommand):
autodetector = MigrationAutodetector
# app1/management/commands/migrate.py
# Assuming "other_app" provides a custom makemigrations command,
# this file is not needed if not
from other_app.management.commands.migrate import (
Command as MigrateCommand,
)
from ._autodetector import MigrationAutodetector
class Command(MigrateCommand):
autodetector = MigrationAutodetector
Note that
Usage
Basic example
-
Create
sql_config.py
module to root of a target app you want to manage custom SQL for. -
Define SQL items in it (
sql_items
), for example:# PostgreSQL example. # Let's define a simple function and let `django_migrate_sql` manage its changes. from django_migrate_sql.config import SQLItem sql_items = [ SQLItem( 'make_sum', # name of the item 'create or replace function make_sum(a int, b int) returns int as $$ ' 'begin return a + b; end; ' '$$ language plpgsql;', # forward sql reverse_sql='drop function make_sum(int, int);', # sql for removal ), ]
-
Create migration
python manage.py makemigrations
:Migrations for 'app_name': 0002_auto_xxxx.py: - Create SQL "make_sum"
You can take a look at content this generated:
from django.db import migrations, models import django_migrate_sql.operations class Migration(migrations.Migration): dependencies = [ ('app_name', '0001_initial'), ] operations = [ django_migrate_sql.operations.CreateSQL( name='make_sum', sql='create or replace function make_sum(a int, b int) returns int as $$ begin return a + b; end; $$ language plpgsql;', reverse_sql='drop function make_sum(int, int);', ), ]
-
Execute migration
python manage.py migrate
:Operations to perform: Apply all migrations: app_name Running migrations: Rendering model states... DONE Applying app_name.0002_xxxx... OK
Check result in
python manage.py dbshell
:db_name=# select make_sum(12, 15); make_sum ---------- 27 (1 row)
Custom types
Now, say, you want to change the function implementation so that it takes a custom type as argument.
-
Edit your
sql_config.py
:# PostgreSQL example #2. # Function and custom type. from django_migrate_sql.config import SQLItem sql_items = [ SQLItem( "make_sum", # name of the item "create or replace function make_sum(a mynum, b mynum) returns mynum as $$ " "begin return (a.num + b.num, 'result')::mynum; end;" "$$ language plpgsql;", # forward sql reverse_sql="drop function make_sum(mynum, mynum);", # sql for removal # depends on `mynum` since takes it as argument. we won't be able to drop function # without dropping `mynum` first. dependencies=[("app_name", "mynum")], ), SQLItem( "mynum", # name of the item "create type mynum as (num int, name varchar(20));", # forward sql reverse_sql="drop type mynum;", # sql for removal ), ]
-
Generate migration
python manage.py makemigrations
:Migrations for 'app_name': 0003_xxxx: - Reverse alter SQL "make_sum" - Create SQL "mynum" - Alter SQL "make_sum" - Alter SQL state "make_sum"
You can take a look at the content this generated:
from django.db import migrations, models import django_migrate_sql.operations class Migration(migrations.Migration): dependencies = [ ('app_name', '0002_xxxx'), ] operations = [ django_migrate_sql.operations.ReverseAlterSQL( name='make_sum', sql='drop function make_sum(int, int);', reverse_sql='create or replace function make_sum(a int, b int) returns int as $$ begin return a + b; end; $$ language plpgsql;', ), django_migrate_sql.operations.CreateSQL( name='mynum', sql='create type mynum as (num int, name varchar(20));', reverse_sql='drop type mynum;', ), django_migrate_sql.operations.AlterSQL( name='make_sum', sql='create or replace function make_sum(a mynum, b mynum) returns mynum as $$ begin return (a.num + b.num, \'result\')::mynum; end; $$ language plpgsql;', reverse_sql='drop function make_sum(mynum, mynum);', ), django_migrate_sql.operations.AlterSQLState( name='make_sum', add_dependencies=[('app_name', 'mynum')], ), ]
NOTE: Previous function is completely dropped before creation because definition of it changed.
CREATE OR REPLACE
would create another version of it, soDROP
makes it clean.If you put
replace=True
as kwarg to anSQLItem
definition, it will NOT drop + create it, but just rerun forward SQL, which isCREATE OR REPLACE
in this example. -
Execute migration
python manage.py migrate
:Operations to perform: Apply all migrations: app_name Running migrations: Rendering model states... DONE Applying brands.0003_xxxx... OK
Check results:
db_name=# select make_sum((5, 'a')::mynum, (3, 'b')::mynum); make_sum ------------ (8,result) (1 row) db_name=# select make_sum(12, 15); ERROR: function make_sum(integer, integer) does not exist LINE 1: select make_sum(12, 15); ^ HINT: No function matches the given name and argument types. You might need to add explicit type casts.
Getting further
For more examples see tests
.
Feel free to open new issues.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file django_migrate_sql_deux-1.1.1.tar.gz
.
File metadata
- Download URL: django_migrate_sql_deux-1.1.1.tar.gz
- Upload date:
- Size: 20.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
f5c79c0c98f3cb71936988c34c8c0ff438c9ce4b02f8b9a8638c0f292d48db98
|
|
MD5 |
79503394368356ea7e887d6d00a937c5
|
|
BLAKE2b-256 |
4038df0b8323fa11f95e30e3c3d2134c47f2c494f88635bb63271a6e72c63f07
|
Provenance
The following attestation bundles were made for django_migrate_sql_deux-1.1.1.tar.gz
:
Publisher:
ci.yml
on browniebroke/django-migrate-sql-deux
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1
-
Predicate type:
https://docs.pypi.org/attestations/publish/v1
-
Subject name:
django_migrate_sql_deux-1.1.1.tar.gz
-
Subject digest:
f5c79c0c98f3cb71936988c34c8c0ff438c9ce4b02f8b9a8638c0f292d48db98
- Sigstore transparency entry: 242895744
- Sigstore integration time:
-
Permalink:
browniebroke/django-migrate-sql-deux@b2eb6769382b26b711ceedf0d49e46193d0bf231
-
Branch / Tag:
refs/heads/main
- Owner: https://github.com/browniebroke
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com
-
Runner Environment:
github-hosted
-
Publication workflow:
ci.yml@b2eb6769382b26b711ceedf0d49e46193d0bf231
-
Trigger Event:
push
-
Statement type:
File details
Details for the file django_migrate_sql_deux-1.1.1-py3-none-any.whl
.
File metadata
- Download URL: django_migrate_sql_deux-1.1.1-py3-none-any.whl
- Upload date:
- Size: 17.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
b94052c82573c07526f042d01dc6630b68ba3e19edf2ce9186426bde84338346
|
|
MD5 |
1894c1085167fa6c88200f6e12d38439
|
|
BLAKE2b-256 |
8ed06d8627bedc6e158d37e337770a42fa30396ca1f4a9026e5ff3140b242851
|
Provenance
The following attestation bundles were made for django_migrate_sql_deux-1.1.1-py3-none-any.whl
:
Publisher:
ci.yml
on browniebroke/django-migrate-sql-deux
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1
-
Predicate type:
https://docs.pypi.org/attestations/publish/v1
-
Subject name:
django_migrate_sql_deux-1.1.1-py3-none-any.whl
-
Subject digest:
b94052c82573c07526f042d01dc6630b68ba3e19edf2ce9186426bde84338346
- Sigstore transparency entry: 242895750
- Sigstore integration time:
-
Permalink:
browniebroke/django-migrate-sql-deux@b2eb6769382b26b711ceedf0d49e46193d0bf231
-
Branch / Tag:
refs/heads/main
- Owner: https://github.com/browniebroke
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com
-
Runner Environment:
github-hosted
-
Publication workflow:
ci.yml@b2eb6769382b26b711ceedf0d49e46193d0bf231
-
Trigger Event:
push
-
Statement type: