Redshift database backend for Django
This product is tested with:
python-2.7, 3.5, 3.6
django-1.8, 1.9, 1.10, 1.11
Differences from postgres_psycopg2 backend
Type mapping:
‘integer identity(1, 1)’ for AutoField
‘bigint identity(1, 1)’ for BigAutoField
‘timestamp with time zone’ for DateTimeField
‘varchar(max)’ for TextField
‘varchar(32)’ for UUIDField
Possibility to multiply VARCHAR length to support utf-8 string, using REDSHIFT_VARCHAR_LENGTH_MULTIPLIER setting.
Stop using:
RETURNING (single insert and bulk insert)
SELECT FOR UPDATE
SELECT DISTINCT ON
SET CONSTRAINTS
INDEX
DEFERRABLE INITIALLY DEFERRED
CONSTRAINT
CHECK
DROP DEFAULT
To support migration:
To add column to existent table on Redshift, column must be nullable
To support modify column, add new column -> data migration -> drop old column -> rename
Please note that the migration support for redshift is not perfect yet.
Note and Limitation
Amazon Redshift doesn’t support RETURNING, so last_insert_id method retrieve MAX(pk) after insertion as a workaround.
refs:
In some case, MAX(pk) workaround does not work correctly. Bulk insertion makes non-contiguous IDs like: 1, 4, 7, 10, … and single insertion after such bulk insertion generates strange id value like 2 (smallest non-used id).
SETTINGS
ENGINE for DATABASES is ‘django_redshift_backend’. You can set the name in your settings.py as:
DATABASES = {
'default': {
'ENGINE': 'django_redshift_backend',
'NAME': '<your database name>',
'USER': '<your database username>',
'PASSWORD': '<your database password>',
'HOST': '<your database hostname>',
'PORT': '5439',
}
}
- REDSHIFT_VARCHAR_LENGTH_MULTIPLIER:
Possibility to multiply VARCHAR length to support utf-8 string. Default is 1.
Using sortkey
There is built-in support for this option for Django >= 1.9. To use sortkey, simply define an ordering on the model meta as follow:
class MyModel(models.Model):
...
class Meta:
ordering = ['col2']
N.B.: there is no validation of this option, instead we let Redshift validate it for you. Be sure to refer to the documentation.
TESTING
Testing this package requires:
tox-1.8 or later
virtualenv-15.0.1 or later
pip-8.1.1 or later
and wheelhouse directory contains psycopg2 manylinux1 wheels for using in each tests.
LICENSE
Apache Software License
CHANGES
0.8.1 (2018-06-19)
#38: Fix 0.8 doesn’t compatible with Python 2. Thanks to Benjy Weinberger.
0.8 (2018-06-01)
Incompatible Changes:
#23,#10 Redshift support time zones in time stamps for migration
IMPORTANT: With this change, the newly created DateTimeField column will be timestamp with timezone (TIMESTAMPTZ) by migration. Therefore, the existing DateTimeField and the new DateTimeField will have different data types as a redshift schema column type. There are no migration feature by django-redshift-backend. see also: https://github.com/shimizukawa/django-redshift-backend/pull/23
New Features:
#20,#26: Support for sortkey. Thanks to Maxime Vdb and Kosei Kitahara.
#24: Add UUIDField support. Thanks to Sindri Guðmundsson.
#14: More compat with redshift: not use SELECT DISTINCT ON.
Bug Fixes:
#15,#21: More compat with redshift: not use CHECK. Thanks to Vasil Vangelovski.
#18: Fix error on migration with django-1.9 or later that raises AttributeError of ‘sql_create_table_unique’.
#27: annotate() does not work on Django-1.9 and later. Thanks to Takayuki Hirai.
Documentation:
Add documentation: http://django-redshift-backend.rtfd.io/
0.7 (2017-06-08)
Drop Python-3.4
Drop Django-1.7
Support Python-3.6
Support Django-1.11
0.6 (2016-12-15)
Fix crush problem when using bulk insert.
0.5 (2016-10-05)
Support Django-1.10
#9: Add support for BigAutoField. Thanks to Maxime Vdb.
Fix crush problem on sqlmigrate when field modified.
0.4 (2016-05-17)
Support Python-3.4 and 3.5
#7: Restore support django-1.7. Version 0.3 doesn’t support django-1.7.
#4: More compat with redshift: not use SET CONSTRAINTS. Thanks to Maxime Vdb.
#6: More compat with redshift: not use sequence reset query. Thanks to Maxime Vdb.
#5: Add REDSHIFT_VARCHAR_LENGTH_MULTIPLIER settings. Thanks to Maxime Vdb.
Support column type changing on migration.
0.3 (2016-05-14)
#3: more compat with Redshift (AutoField, DateTimeField, Index). Thanks to Maxime Vdb.
More compat with redshift: add TextField
More compat with redshift: not use DEFERRABLE, CONSTRAINT, DROP DEFAULT
More compat with redshift: support modify column
0.2.1 (2016-02-01)
“SET TIME_ZONE” warning is changed as debug log for ‘django.db.backend’ logger.
0.2 (2016-01-08)
Disable “SET TIME_ZONE” SQL execution even if settings.TIME_ZONE is specified.
0.1.2 (2015-06-5)
Support Django-1.8
0.1.1 (2015-03-27)
Disable “SELECT FOR UPDATE” SQL execution.
0.1 (2015-03-24)
Support Django-1.7
Support “INSERT INTO” SQL execution without “RETURNING” clause.
Release files for django-redshift-backend 0.8.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| django-redshift-backend-0.8.1.tar.gz | 9.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| django_redshift_backend-0.8.1-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details |
Total release size: 22.9 kB
Release files / django-redshift-backend-0.8.1.tar.gz
| Download URL | django-redshift-backend-0.8.1.tar.gz |
|---|---|
| Size | 9.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c6fdd7ed54c51beabd2c7caf99841100aa1c98406cfd7b5270fa0fd3155c424f
|
|
BLAKE2b-256 checksum How to use checksums |
87332b2783a1897e7d685a2349d1d0de551b0290f5d0e58aa9a845a06b862b75
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / django_redshift_backend-0.8.1-py2.py3-none-any.whl
| Download URL | django_redshift_backend-0.8.1-py2.py3-none-any.whl |
|---|---|
| Size | 13.3 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
89d20e57d3830752243cee1c2a88fe1e42c0553a78fe20bf42e04325f0dca2d5
|
|
BLAKE2b-256 checksum How to use checksums |
4299e7e535d6f33cfee91461ed6cc397b6c96f7c27a386d61fd24ae17278505a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |