Django Secured Fields

Django encrypted fields with search enabled.

Requirements

• Python 3.10+
• Django 4.2+
• PostgreSQL, MySQL or SQLite

Features

• Automatically encrypt/decrypt field value using cryptography's Fernet
• Built-in search lookup on the encrypted fields from hashlib's SHA-256 hash value. in and isnull lookup also supported.
• Supports most of available Django fields including BinaryField, JSONField, and FileField.

Installation

pip install django-secured-fields

Setup

1. Add secured_fields into INSTALLED_APPS

   # settings.py
   
   INSTALLED_APPS = [
       ...
       'secured_fields',
   ]
   

2. Generate a new key using for encryption

   $ python manage.py generate_key
   KEY: TtY8MAeXuhdKDd1HfGUwim-vQ8H7fXyRQ9J8pTi_-lg=
   HASH_SALT: 500d492e
   

3. Put generated key(s) and hash salt in settings

   # settings.py
   
   SECURED_FIELDS_KEY = 'TtY8MAeXuhdKDd1HfGUwim-vQ8H7fXyRQ9J8pTi_-lg='
   # or multiple keys for rotation
   SECURED_FIELDS_KEY = [
       'TtY8MAeXuhdKDd1HfGUwim-vQ8H7fXyRQ9J8pTi_-lg=',
       '...',
   ]
   
   # optional
   SECURED_FILDS_HASH_SALT = '500d492e'
   

Usage

Simple Usage

# models.py
import secured_fields

phone_number = secured_fields.EncryptedCharField(max_length=10)

Enable Searching

# models.py
import secured_fields

id_card_number = secured_fields.EncryptedCharField(max_length=18, searchable=True)

Supported Fields

• EncryptedBinaryField
• EncryptedBooleanField
• EncryptedCharField
• EncryptedDateField
• EncryptedDateTimeField
• EncryptedDecimalField
• EncryptedFileField
• EncryptedImageField
• EncryptedIntegerField
• EncryptedJSONField
• EncryptedTextField

Settings

APIs

Field Arguments

Changing searchable on a field with existing records

Existing records stay readable after changing searchable, but they keep the storage format of the previous flag until they are re-saved:

• False → True: existing records do not have a hashed section yet, so exact/in lookups will not match them.
• True → False: existing records still carry the old hashed section — a deterministic fingerprint of the value — until they are re-saved.

After changing the flag (and running makemigrations/migrate, since the database index changes), re-save the affected records to convert them to the new format. Do this in a data migration that runs after the generated AlterField operation — the historical model must already carry the new searchable value, otherwise the records are silently re-written in the old format:

def resave_records(apps, schema_editor):
    model = apps.get_model('myapp', 'MyModel')
    model.objects.bulk_update(model.objects.all(), ['my_field'], batch_size=1000)

bulk_update re-encrypts through the same path as save() but batches the queries, and — unlike re-saving each record — does not overwrite auto_now date/datetime fields with the migration run time.

Encryption

> from secured_fields.fernet import get_fernet

> data = b'test'

> encrypted_data = get_fernet().encrypt(data)
> encrypted_data
b'gAAAAABh2_Ry_thxLTuFFXeMc9hNttah82979JPuMSjnssRB0DmbgwdtEU5dapBgISOST_a_egDc66EG_ZtVu_EqF_69djJwuA=='

> get_fernet().decrypt(encrypted_data)
b'test'

Rotate Keys

> from secured_fields.fernet import get_fernet

> encrypted_data = get_fernet().encrypt(b'test')
> encrypted_data
b'gAAAAABh2_Ry_thxLTuFFXeMc9hNttah82979JPuMSjnssRB0DmbgwdtEU5dapBgISOST_a_egDc66EG_ZtVu_EqF_69djJwuA=='

> rotated_encrypted_data = get_fernet().rotate(encrypted_data)
> get_fernet().decrypt(rotated_encrypted_data)
b'test'

See more details in MultiFernet.rotate.

EncryptedMixin

If you have a field which is not supported by the package, you can use EncryptedMixin to enable encryption and search functionality for that custom field.

import secured_fields
from django.db import models

class EncryptedUUIDField(secured_fields.EncryptedMixin, models.UUIDField):
    pass

task_id = EncryptedUUIDField(searchable=True)

EncryptedStorageMixin

If you use a custom file storage class (e.g. defined in settings.py's STORAGES), you can enable file encryption using EncryptedStorageMixin.

import secured_fields
from minio_storage.storage import MinioMediaStorage

class EncryptedMinioMediaStorage(
    secured_fields.EncryptedStorageMixin,
    MinioMediaStorage,
):
    pass

Known Limitation

• in lookup on JSONField is not available
• Large files are not performance-friendly at the moment (see #2)
• Search on BinaryField does not supported at the moment (see #6)
• Changing searchable on a field with existing records requires re-saving the records to make search results consistent (see Changing searchable on a field with existing records)
• update() only accepts plain values or Value() for encrypted fields; other expressions such as F(), Case or database functions raise ExpressionNotSupported, since their result would be written unencrypted (see #52). bulk_update() is supported

Development

Requirements

• Docker
• Poetry 2.0+
• MySQL Client
  • brew install mysql-client
  • echo 'export PATH="/usr/local/opt/mysql-client/bin:$PATH"' >> ~/.bash_profile

Running Project

1. Start backend databases

   make up-db
   

2. Run tests (see: Testing)

Linting

make lint

Testing

make test-pg  # or make test-mysql, make test-sqlite

Fix Formatting

make yapf