Mind Castle - Build a wall around your secrets

A universal store for your secret data. Don't delay securing you or your customer's data by deliberating over cloud secret stores. Mind Castle makes it easy to get started, and easy to switch between cloud secret stores.

Mind Castle currently supports:

• AWS KMS
• Local Encryption
• None (passthrough - Mind Castle doesn't modify your data)

Architecture

Mind Castle comes in three parts:

• A unified interface for several secret stores.
• An SQLAlchemy column type, and a Django model field, that transparently store and retrieve secrets for you.
• A migration tool to convert your existing DB column data into secrets.

Some other notes:

• Mind Castle is configured and secret stores are initialised at import time. That means env-vars used for configuration need to be defined when Mind Castle is imported.
• Mind Castle makes no attempt to manage secrets in memory. Memory management in Python is futile, and if you need that level of control it's best to use another language.

Install

pip install mind-castle

Configure

You can configure Mind Castle by setting environment variables for your chosen secret store. To see what configuration options are required for each store:

$ python -m mind_castle
╭────────────────────────────────────────── MIND CASTLE ───────────────────────────────────────────╮
│                                                                                                  │
│                                          Secret Stores                                           │
│ ┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │
│ ┃ Store Type      ┃ Required env var                    ┃ Optional env var                     ┃ │
│ ┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │
│ │ localencryption │ MIND_CASTLE_LOCAL_ENCRYPTION_SECRET │                                      │ │
│ │                 │                                     │                                      │ │
│ │ awskms          │ MIND_CASTLE_AWS_REGION              │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE    │ │
│ │                 │ MIND_CASTLE_AWS_ACCESS_KEY_ID       │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_T… │ │
│ │                 │ MIND_CASTLE_AWS_SECRET_ACCESS_KEY   │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_C… │ │
│ │                 │ MIND_CASTLE_AWS_KMS_KEY_ARN         │                                      │ │
│ │                 │ --- OR ---                          │                                      │ │
│ │                 │ MIND_CASTLE_AWS_REGION              │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE    │ │
│ │                 │ MIND_CASTLE_AWS_USE_ENV_AUTH        │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_T… │ │
│ │                 │ MIND_CASTLE_AWS_KMS_KEY_ARN         │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_C… │ │
│ │                 │                                     │                                      │ │
│ └─────────────────┴─────────────────────────────────────┴──────────────────────────────────────┘ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

"none" is a fourth store type you can pass to a column or field. It is a passthrough handled by the column type itself rather than a store, so it takes no configuration and does not appear in that table. Anything else raises ValueError: Trying to put secret to unknown/unconfigured store '<name>' the first time you save.

AWS KMS data-key cache (optional)

The awskms store uses envelope encryption: each secret is sealed with its own data key, which KMS must decrypt on every read. When the same secret is read repeatedly (e.g. frequent full re-fetches), this drives repeated KMS Decrypt calls. You can opt in to caching the decrypted data keys in-process:

Notes:

• Caching applies to decryption only; create_secret still generates a fresh data key per secret. Legacy non-envelope secrets are never cached.
• A revoked/rotated KMS key keeps working from cache until the TTL expires — keep the TTL short. Invalid or non-positive TTL/capacity disables the cache.
• The cache is per-process (each worker has its own) and holds decrypted data keys in memory (up to CAPACITY, until TTL). Config is read at startup; changing it requires a restart.

Use

Migrating Existing Data

A migration tool is provided to move your existing data into a secret store. The tool assumes you have a database that can be connected to using SQLAlchemy and create_engine(<db_uri>).

You will need to configure your selected secret store using environment variables as described above. The migration tool depends on iterfzf, which isn't installed by the base package - install it with pip install "mind-castle[migrate]".

You can see what other options the migration tool accepts with:

$ python migration_tool.py --help

 Usage: migration_tool.py [OPTIONS] DB_URI

╭─ Arguments ─────────────────────────────────────────────────────────────────────────╮
│ *    db_uri      TEXT  [default: None] [required]                                   │
╰─────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────╮
│ --target-table                        TEXT  [default: None]                         │
│ --target-column                       TEXT  [default: None]                         │
│ --dry-run           --no-dry-run            [default: dry-run]                      │
│ --demigrate         --no-demigrate          [default: no-demigrate]                 │
│ --to-secret-type                      TEXT  [default: json]                         │
│ --help                                      Show this message and exit.             │
╰─────────────────────────────────────────────────────────────────────────────────────╯

The easiest thing to do is specify --to-secret-type and let the tool guide you through the rest. e.g.:

$ MIND_CASTLE_LOCAL_ENCRYPTION_SECRET="<your_base64_key>" python migration_tool.py "postgresql://<user>:<pass>@<host>:5432/<database>" --to-secret-type localencryption

The tool will list tables and columns in the selected DB for you, and ask you to select each. Once a dry run is complete and you also have a backup of your database, you can add --no-dry-run to migrate the data for real.

SQLAlchemy Model

Once any existing data is migrated, you can update your SQLAlchemy model to include a SecretData column:

from mind_castle.sqlalchemy_type import SecretData

class MyDBModel(Base):
    name = Column(String, nullable=False)
    created_at = Column(DateTime, default=datetime.datetime.now)
    secret_data = Column(SecretData("localencryption"))

Your secrets will then be safely stored in AWS (or encrypted locally, or anywhere else you like)! The storage and retrieval of secrets will be completely transparent to your application.

Django Model

For a Django model, use mind_castle.django_type.SecretData instead - it's a JSONField subclass with the same encrypt/decrypt behavior. Install the django extra (pip install "mind-castle[django]") first.

from mind_castle.django_type import SecretData

class MyDBModel(models.Model):
    name = models.CharField(max_length=255)
    created_at = models.DateTimeField(auto_now_add=True)
    secret_data = SecretData("localencryption", null=True)

Notes specific to the Django field:

• Filtering or looking up by value (MyDBModel.objects.filter(secret_data=...), secret_data__somekey=...) raises FieldError on any store except "none" - the stored value is ciphertext with a fresh nonce per encryption, so a value-equality comparison could otherwise silently match nothing. Null checks still work, written either way: filter(secret_data=None) and filter(secret_data__isnull=True).
• Admin and ModelForm rendering show the decrypted value, matching a normal JSONField's editable UX. dumpdata/serializers.serialize() always export the encrypted envelope, and loaddata reads that envelope back without re-encrypting it, so a fixture round-trips.

How values are compared, changed and moved around

These three rules apply to both the SQLAlchemy and the Django field types.

Changing a value

Reassigning the field and editing it in place both work:

obj.secret_data = {**obj.secret_data, "key": "new"}   # reassign
obj.secret_data["key"] = "new"                        # edit in place

A decrypted value is handed back inside a wrapper that keeps a copy of what it decrypted, so an edit to the contents is still noticed and the value is re-encrypted on save.

One caveat on SQLAlchemy: editing in place does not mark the attribute dirty, which is a general SQLAlchemy behaviour and not specific to this type. Nothing is written unless something else on that row changes, or you call flag_modified(obj, "secret_data"). Django writes every field on save(), so it needs nothing extra.

Pickling

Pickling a decrypted value, or a whole model instance holding one, writes the encrypted envelope into the pickle and decrypts it again on load. That is what lets an instance cross a process or network boundary (Django's cache framework, a Celery task argument, multiprocessing) without the plaintext travelling. The loading process needs the matching store configured or the load raises. Pickling a value that was edited since it was decrypted also raises, because the envelope would still hold the old one.

What counts as an encrypted value

On any store except "none", a stored dict carrying both a mind_castle_secret_type and an encrypted_value key is treated as ciphertext, and one that does not is passed through as plaintext. That single rule is used everywhere a value crosses a boundary: reading a column, loading a fixture, unpickling. A dict that claims to be an envelope but does not parse as one is a corrupt row and raises, rather than being handed back as if it were your data.

TODO

• Make migration script work for non-json columns
• Support deleting secrets when row is deleted
• Implement prefixes/folders for secrets
• Explain how secrets are stored
• Enforce tests on PR / branch protections
• Wire secret_key_prefix through to the stores, or drop it from the signature