Kist

Keep It Simple and Trusted.
Git-governed encrypted secret storage for Python applications.

Status · Quick start · How it works · Git workflow · CI/CD & GitOps · Security boundaries · License

Alpha software — Kist 0.1.0. Kist is suitable for evaluation and controlled self-hosted deployments. Its API, database schema, and encrypted-envelope format may evolve before a stable release. Review the code, test recovery procedures, and do not treat a local Git hook as an organisational security control.

Kist is a Python library and CLI for teams that want secret changes reviewed in Git, stored encrypted in PostgreSQL, and retrieved by Python applications—without operating a separate Vault cluster.

Kist means Keep It Simple and Trusted.

It is built for teams that want encrypted secrets and a reviewable Git workflow without turning secret storage into a second full-time infrastructure project. The model stays deliberately small: a profile password, authenticated encryption, PostgreSQL, and clear operational boundaries.

What Kist does

• Encrypts JSON-compatible secret values with Argon2id-derived AES-256-GCM keys.
• Stores encrypted values in PostgreSQL, scoped by organisation, project, and profile.
• Locks nested YAML mappings into dot-separated secret keys while preserving JSON-compatible types.
• Unlocks a profile back into readable YAML when the correct password is available.
• Uses Git clean/smudge filters to keep selected YAML secret files encrypted in Git while exposing plaintext only in the authorised working tree.
• Provides an optional local pre-commit guard that detects plaintext in configured Kist secret paths and common credential patterns.
• Integrates with CI/CD pipelines (GitHub Actions, GitLab CI/CD, Bitbucket Pipelines) for automated PR leak scanning and continuous secret synchronization (GitOps).
• Supports bounded in-memory caches and PostgreSQL LISTEN / NOTIFY invalidation through the optional runtime store helpers.

What Kist does not do

Kist 0.1.0 deliberately stays small:

• It does not provide the sharks surrounding the island in the image above (you will have to supply your own biological perimeter defense).
• It is not a replacement for HashiCorp Vault, a cloud secret manager, or a full identity and governance platform.
• It does not include Kist Guard, cloud KMS, HSM, or managed key-release support yet.
• It does not protect a secret after an authorised application has decrypted it.
• It does not make a local Git hook impossible to bypass (git commit --no-verify will always remain an effective way to shoot oneself in the foot).
• It does not replace key backup, password-sharing procedures, CI access policy, incident response, or the people responsible for operating them.

Kist reduces moving parts in a secret workflow; it does not replace operational judgement, incident response, or the uncomfortable meeting after a leaked credential.

Status

Kist is currently an alpha project.

The 0.1.0 release supports only the password-v1 encryption mode:

profile password
    + random salt
    ↓ Argon2id
AES-256 key
    + random nonce + authenticated context
    ↓ AES-256-GCM
encrypted Kist envelope
    ↓
PostgreSQL or Git

Planned but not included in 0.1.0:

KMS data-encryption keys
Kist Guard integration
Cloud KMS providers
Managed hosting
Key rotation and migration tooling

Installation

Install from PyPI with uv or pip:

uv add kist

pip install kist

Check that the package and CLI are available:

python -c "import kist; print(kist.__version__)"
python -m kist --help
kist --help

Quick start

1. Configure Kist

For local development, configure PostgreSQL and a profile password. Never commit these values to Git.

export KIST_ENV=development
export KIST_TLS_MODE=disable

export KIST_DATABASE_URL='postgresql://kist_user:replace-me@127.0.0.1:5432/kist'
export KIST_PASSWORD='replace-with-a-long-random-password'

# Optional scope values.
export KIST_ORG_ID='local'
export KIST_PROJECT_ID='default'
export KIST_PROFILE='default'

For a remote production PostgreSQL service, use TLS with certificate verification:

export KIST_ENV=production
export KIST_TLS_MODE=verify-full
export PGSSLROOTCERT='/run/secrets/postgres-ca.pem'

export KIST_DATABASE_URL='postgresql://kist_user:replace-me@db.example.internal:5432/kist?sslmode=verify-full'
export KIST_PASSWORD='provided-by-your-runtime-secret-injection-mechanism'

sslmode=require encrypts transport but does not verify the database server identity. Prefer verify-full for remote production databases.

2. Initialise the database schema

Schema setup is explicit. Run it using a controlled deployment/setup identity:

kist init-db

A normal runtime application should ideally have only the database permissions required to read and write Kist data; it should not need permission to create tables.

3. Store and retrieve a value

kist set example.api_key 'example-value'
kist get example.api_key

Output:

example-value

Avoid placing real production secrets directly on a shell command line: terminal history, process inspection, and CI logs can all turn a convenient command into an inconvenient incident report.

4. Use Kist from Python

import os

from kist import Kist, KistScope

# Automatic resolution: checks KIST_PASSWORD_<PROFILE> -> KIST_PASSWORD -> ~/.kist/pwd_<profile>
store = Kist(
    profile_name=os.getenv("KIST_PROFILE", "default"),
    db_url=os.environ.get("KIST_DATABASE_URL"),
    scope=KistScope.from_environment(),
)

# Alternatively, pass master_password explicitly:
# store = Kist(profile_name="production", master_password=os.environ["KIST_PASSWORD_PRODUCTION"])

api_key = store.get("example.api_key")
print(api_key)

The public 0.1.0 API is intentionally small:

from kist import (
    DecryptionError,
    InvalidEnvelopeError,
    Kist,
    KistError,
    KistScope,
    MissingKeyError,
)

How it works

Password-v1 envelopes

Kist stores an encrypted binary envelope, not a plaintext secret value.

plaintext value
    ↓
JSON serialisation
    ↓
AES-256-GCM encryption
    ↑
Argon2id(profile password, random salt)
    ↑
profile password

Each envelope contains, conceptually:

Component	Purpose	Secret?
Magic/version fields	Identifies the Kist format and supported algorithms	No
Argon2id parameters	Allows the same password-derived key to be recreated	No
Salt	Makes each password-key derivation distinct	No
Nonce	Makes every AES-GCM encryption distinct	No
Ciphertext and authentication tag	Stores the protected secret value	Protected, but safe to store
Context commitment	Binds the envelope to its intended scope and identity	No

The salt is used before encryption to derive a key from a password. The nonce is used for every AES-GCM encryption. Both are stored with the ciphertext; neither is a secret. The important rule is that Kist uses a fresh nonce for every encryption under a key.

Authenticated context

Kist binds each database value to authenticated context that includes:

organisation ID
project ID
profile ID
profile name
logical secret key
secret version

That means an encrypted value should not successfully decrypt if it is copied to a different profile, secret key, scope, or version.

For example, ciphertext created for:

local/default/production/database.password/version-4

cannot simply be relabelled as:

local/default/production/payment.api_key/version-4

and accepted by Kist. The authentication check fails closed.

YAML lock and unlock

Kist accepts nested YAML mappings and flattens them into dot-separated keys.

# secrets_production.yaml
sql:
  pg:
    host: db.internal
    port: 5432
    ssl_enabled: true
third_party:
  stripe:
    api_key: sk_example_value

This becomes values similar to:

sql.pg.host                    → "db.internal"
sql.pg.port                    → 5432
sql.pg.ssl_enabled             → true
third_party.stripe.api_key     → "sk_example_value"

JSON-compatible types are preserved. Kist uses yaml.safe_load; YAML roots must be mappings and mapping keys must be non-empty strings without dots.

Lock a YAML file into PostgreSQL:

kist lock secrets/production.yaml

Explicitly specify or override a profile:

kist --profile production lock secrets/production.yaml

Defense-in-Depth Safety: By default, kist lock verifies that the active profile matches the target file basename (e.g. --profile production expects production.yaml or secrets/production.yaml). Accidental mismatches are blocked to prevent cross-profile data corruption. To explicitly load a differently named file for custom workflows, use --allow-mismatch or --force:

kist --profile production lock secrets/custom_seed.yaml --allow-mismatch

Prune database values that no longer appear in the YAML source:

kist lock secrets/production.yaml --prune

Destructive Flag Protection: In interactive terminals, --prune prompts for explicit user confirmation before permanently deleting obsolete database rows. In automated pipelines or scripted workflows, supply --force or --yes to proceed without interactive confirmation.

Reconstruct a profile as YAML:

kist unlock secrets/production.yaml

Writer model & concurrency: Kist serializes concurrent profile writes (such as overlapping CI deployment runs) using PostgreSQL transaction-level advisory locks (pg_advisory_xact_lock), and protects programmatic single-key updates (kist.set) with row-level locks (SELECT ... FOR UPDATE) and automatic retry loops.

Runtime cache invalidation

When Kist changes a value, it sends a PostgreSQL notification on the kist_secret_updated channel. DynamicSecretStore and AsyncDynamicSecretStore can listen for these events and clear the affected in-memory cache entry.

Git/deployment writer updates Kist database
    ↓
PostgreSQL NOTIFY
    ↓
running application cache invalidates the affected key
    ↓
next read decrypts the current stored value

This is cache invalidation, not secret telepathy: each application still needs its configured database access and profile password to retrieve a value.

Git workflow

Kist integrates with Git clean/smudge filters for selected YAML secret files.

authorised working tree       Git index / repository
-----------------------       ----------------------
plaintext secrets.yaml
        │
        ├── git add ── clean filter ──> encrypted Kist envelope
        │
        └── git checkout <── smudge filter <── encrypted Kist envelope
                                      │
                                      └── plaintext working-tree file

Initialise a repository

Inside a Git work tree:

kist init

Kist configures local clean/smudge filters, appends missing Kist rules to .gitattributes, adds .kist/ to .gitignore, and installs a local pre-commit guard when safe to do so.

Expected Kist attribute rules include:

secrets*.yaml filter=kist
secrets*.yml filter=kist
secrets/*.yaml filter=kist
secrets/*.yml filter=kist

Kist will not silently overwrite an unrelated pre-commit hook. Review the hook first, then use the explicit force option only when appropriate.

Check and repair local setup

kist diagnose
kist fix
kist fix --apply
kist status

kist fix is a dry run. kist fix --apply repairs only local Git integration. It does not create profile keys, restage files, or modify encrypted secret values.

Local hook limits

The local pre-commit guard blocks likely plaintext credentials and plaintext files in Kist-managed secret paths before a normal commit. It is useful fast feedback, but it is not an absolute Git leak guard.

Any developer who controls a local clone can bypass a client-side hook, for example with:

git commit --no-verify

Treat local hooks as a seatbelt, not an indestructible force field.

For organisation-wide enforcement:

1. Run an equivalent secret scanner in required CI.
2. Protect production/default branches.
3. Require merge requests and passing security checks.
4. Restrict who can modify CI configuration and security policy.
5. Where you administer the Git server, consider a server-side pre-receive scanner that rejects pushes before refs are updated.

CI/CD integration & GitOps

Kist turns Git into a secure, reviewable control plane for application secrets (GitOps for Secrets) while keeping raw credentials strictly outside of unencrypted Git history and log streams.

┌───────────────────────────────┐     Pull Request     ┌────────────────────────────────────┐
│ Developer pushes branch       │ ───────────────────> │ CI Server: PR Leak Guard           │
│ (secrets encrypted by filter) │                      │ - Verifies valid Kist envelopes    │
└───────────────────────────────┘                      │ - Blocks unencrypted files/tokens  │
                                                       └─────────────────┬──────────────────┘
                                                                         │ Approved & Merged
                                                                         ▼
┌───────────────────────────────┐     PostgreSQL       ┌────────────────────────────────────┐
│ Application Runtime Containers│ <─────────────────── │ CI/CD Server: Secret Sync          │
│ - DynamicSecretStore caches   │    LISTEN/NOTIFY     │ - Runs `kist lock secrets.yaml`    │
│ - Instant auto-invalidation   │                      │ - Updates database ciphertext      │
└───────────────────────────────┘                      └────────────────────────────────────┘

1. Server-side PR leak protection

While local pre-commit hooks provide fast feedback for developers, server-side CI checks guarantee that no unencrypted secret file or exposed credential ever merges into protected branches (e.g., main, staging).

In Pull Requests / Merge Requests, CI verifies that all tracked secrets*.yaml files are valid kist-v1 binary envelopes:

- name: Verify Secret Envelope Encryption
  run: |
    python - << 'EOF'
    import sys
    from pathlib import Path
    from kist.core import is_kist_envelope

    secret_files = list(Path('.').glob('secrets*.yaml')) + list(Path('.').glob('secrets*.yml'))
    for file_path in secret_files:
        if file_path.is_file() and file_path.stat().st_size > 0:
            if not is_kist_envelope(file_path.read_bytes()):
                print(f"🚨 Plaintext secret leak detected in {file_path}!", file=sys.stderr)
                sys.exit(1)
    print("✅ All secret files verified as encrypted Kist envelopes.")
    EOF

2. Automated secret deployment & live reload

When a reviewed PR merges to main, CI securely synchronizes the updated secrets to your centralized PostgreSQL database:

# In CI with KIST_DATABASE_URL and KIST_PASSWORD_PRODUCTION configured:
kist --profile production lock secrets_production.yaml

When secrets are locked into PostgreSQL, Kist emits a NOTIFY kist_updates event. Applications running DynamicSecretStore or AsyncDynamicSecretStore receive the signal in real time and automatically refresh cached values without container restarts or downtime.

3. Production-ready CI templates

Standalone, copy-paste templates are available in the ressources/ directory:

• GitHub Actions (ressources/github/):
  • leak-guard.yml: PR scan and leak prevention.
  • deploy.yml: PostgreSQL secret sync on merge to main.
• GitLab CI/CD (ressources/gitlab/):
  • .gitlab-ci.yml: Multi-stage pipeline with security scanning, secret sync, and deployment.
• Bitbucket Pipelines (ressources/bitbucket/):
  • bitbucket-pipelines.yml: Step definitions for PR leak guard and automated deployment.

For detailed setup instructions and variable configuration, see the CI/CD Resources Guide.

Password resolution and local profile keys

For any profile named <name>, Kist resolves the required password in the following order of precedence:

1. Profile-Specific Environment Variable: KIST_PASSWORD_<NAME_UPPERCASE> (e.g., KIST_PASSWORD_PRODUCTION, KIST_PASSWORD_STAGING, KIST_PASSWORD_DEFAULT)
2. Global Fallback Environment Variable: KIST_PASSWORD (used across profiles if no specific variable is defined)
3. Local Key File: ~/.kist/pwd_<name> (e.g., ~/.kist/pwd_production, ~/.kist/pwd_staging, ~/.kist/pwd_default)

Managing multiple profiles with the CLI

When running CLI commands with the --profile flag, Kist automatically resolves the corresponding profile password:

# 1. Using profile-specific environment variables (recommended for CI/CD & Cloud)
export KIST_PASSWORD_PRODUCTION='super-secret-random-production-key'
export KIST_PASSWORD_STAGING='staging-specific-random-key'

kist --profile production lock secrets_production.yaml
kist --profile staging status

# 2. Using local key files (recommended for local development)
kist profile create development
kist profile create staging

kist --profile staging status

Create a local profile password explicitly:

kist profile create production

Kist intentionally does not generate profile keys implicitly during normal operations. A surprise key is a fine way to create a surprise recovery problem.

On POSIX systems, Kist uses owner-only protections for local key material:

~/.kist/                 0700
~/.kist/pwd_<profile>    0600

Keep local profile keys out of Git, backups with broad access, screenshots, and chat messages. Prefer runtime-injected environment variables or a purpose-built secret delivery mechanism for deployed applications.

Python runtime stores

Kist includes optional bounded in-memory cache wrappers in kist.stores.

from kist.stores import TTLSecretStore

store = TTLSecretStore(
    profile_name="production",
    ttl_seconds=30,
)

api_key = store.get("third_party.stripe.api_key")

For PostgreSQL notification-driven invalidation:

from kist.stores import DynamicSecretStore

store = DynamicSecretStore(
    profile_name="production",
    auto_start=True,
)

try:
    api_key = store.get("third_party.stripe.api_key")
finally:
    store.close()

For asynchronous applications:

from kist.stores import AsyncDynamicSecretStore

async def use_secrets() -> None:
    async with AsyncDynamicSecretStore(
        profile_name="production",
    ) as store:
        api_key = await store.get("third_party.stripe.api_key")

Caching warning

Caching makes reads faster by retaining plaintext in process memory for a bounded period. Use it only when that trade-off is acceptable for your application. Set an explicit cache policy or disable caching if you need every read to go back through Kist.

Security boundaries

Kist is deliberately explicit about what it protects and what it does not.

Kist helps protect against

• Accidental storage of plaintext secret values in the Kist PostgreSQL tables.
• Accidental storage of configured Kist YAML files as plaintext Git blobs when filters are installed and used normally.
• Undetected modification or substitution of encrypted values across authenticated Kist scope/context.
• Simple reuse of a database dump or Git blob without the corresponding profile password.
• Stale in-process cache values after a Kist update, when notification invalidation is configured and connected.

Kist does not protect against

• A compromised authorised application process that can legitimately retrieve a secret.
• Malware, debug tooling, crash dumps, or hostile dependencies reading application memory.
• A leaked profile password.
• A developer intentionally bypassing or removing a local Git hook.
• A Git server, CI system, database administrator, or operating-system administrator compromised at a higher trust level.
• Deletion, rollback, availability failures, or backup loss unless your infrastructure handles them.
• Business-sensitive metadata in logical secret names, project names, or environment variables.

Do not put plaintext secret values, bearer tokens, passwords, or private keys into:

Git commit messages
CI logs
issue trackers
URLs or query strings
KMS/encryption context
error messages
screenshots

Database model

Kist 0.1.0 uses PostgreSQL tables conceptually organised as:

kist_organizations
    └── kist_projects
            └── kist_profiles
                    └── kist

The kist table stores:

profile ID
logical secret key
encrypted binary envelope
crypto mode: password-v1
crypto version
logical secret version
update timestamp

The database stores encrypted values, not profile passwords or plaintext values. The profile password remains external to PostgreSQL.

For production, use separate roles:

migration/setup role
→ creates or migrates schema

runtime application role
→ reads/writes only required Kist data
→ does not have broad DDL permissions

Troubleshooting

MissingKeyError or “No encryption key configured”

Set the relevant profile environment variable:

export KIST_PASSWORD_PRODUCTION='...'

or create an explicit local key:

kist profile create production

PostgreSQL connection fails

Check that the URL points to a running PostgreSQL server and that local TLS settings are intentional:

pg_isready -h 127.0.0.1 -p 5432

For a local non-TLS development server:

export KIST_ENV=development
export KIST_TLS_MODE=disable

For production remote PostgreSQL, use verify-full and provide a trusted CA certificate.

Git checkout refuses to decrypt a secret file

Check that the correct profile password is available and that the repository/path/scope context matches the one used when the file was encrypted:

kist status
kist diagnose

Kist fails closed when the password, encrypted content, profile, scope, or path does not match.

Kist pre-commit guard blocks a commit

First, assume the warning is useful. Check the staged content:

git diff --cached

If a real credential was staged in plaintext, remove it, rotate it if needed, and use the approved Kist workflow. If you believe it is a false positive, adjust policy deliberately rather than treating --no-verify as a normal workflow.

Development and release notes

Kist 0.1.0 is designed to be understandable before it is expansive. If you find a security issue, report it privately through the repository’s security-advisory process rather than opening a public issue with exploit details.

Before using Kist for sensitive production workloads, review:

• profile-password generation, storage, backup, and rotation;
• PostgreSQL authentication, authorisation, network exposure, and backups;
• TLS verification for remote database connections;
• CI/server-side secret scanning and protected branch policy;
• runtime process security and logging;
• recovery procedures after a lost profile password or leaked credential.

License

Kist is licensed under the Apache License, Version 2.0.

Apache-2.0 is a permissive license: you may use, modify, redistribute, and use Kist commercially, subject to the license terms and preservation of applicable notices. It includes an explicit patent grant from contributors.

Cryptographic primitives (AES-256-GCM AEAD and Argon2id KDF) are powered by the standard Python Cryptographic Authority (cryptography) library.

See LICENCE and NOTICE.

---

Keep It Simple and Trusted.
Encrypt the value. Review the change. Protect the boundary. Keep the drama for production incidents you did not have.