hivemind-redis-database

Redis-backed client database plugin for hivemind-core.

This plugin implements the hivemind-plugin-manager AbstractDB contract on top of Redis. Use it for multi-host fleets, high-availability setups, and deployments where several hivemind-core processes share one database.

It supports:

• single Redis
• Redis Cluster
• optional TLS
• optional RediSearch / Redis Stack acceleration
• transactional same-slot writes in cluster mode through cluster_hash_tag
• repair and migration tooling for production rollouts

Why This Backend

This backend keeps HiveMind client records in Redis with secondary indexes for name and api_key.

Key operational points:

• it auto-detects single Redis vs Redis Cluster when you point host and port at a live node
• it uses RediSearch when the module is available, and falls back to normal Redis indexing when it is not
• it keeps exact-match search semantics whether or not RediSearch is enabled
• it hides revoked clients from iteration and search results
• it exposes sync() to rebuild counters, set indexes, and search hashes after interrupted writes or manual Redis changes

Installation

pip install hivemind-redis-database

Compatibility:

• Python 3.10+
• Redis for single-instance mode
• Redis Cluster for cluster mode
• Redis Stack, or Redis with the RediSearch module loaded, for advanced search

Deployment Modes

Recommendation:

• single Redis deployments: upgrade directly, no migration needed
• new Redis Cluster deployments: enable cluster_hash_tag from day one
• existing Redis Cluster deployments: upgrade the legacy namespace first if needed, then migrate to cluster_hash_tag in a maintenance window

HiveMind-core Configuration

HiveMind-core loads this plugin from server.json and passes the plugin config through directly.

Important note:

• name and subfolder are part of the HiveMind-core database contract
• index_prefix controls the actual Redis key namespace this backend uses
• this backend accepts subfolder for compatibility, but it does not affect Redis keys or data layout

Single Redis

{
  "database": {
    "module": "hivemind-redis-db-plugin",
    "hivemind-redis-db-plugin": {
      "name": "clients",
      "host": "127.0.0.1",
      "port": 6379,
      "db": 1,
      "password": "",
      "index_prefix": "client",
      "max_connections": 10
    }
  }
}

Redis Cluster, Recommended Mode

{
  "database": {
    "module": "hivemind-redis-db-plugin",
    "hivemind-redis-db-plugin": {
      "name": "clients",
      "cluster_nodes": [
        {"host": "redis-node1", "port": 6379},
        {"host": "redis-node2", "port": 6379},
        {"host": "redis-node3", "port": 6379}
      ],
      "cluster_hash_tag": "clients",
      "password": "your_password",
      "index_prefix": "client",
      "max_connections": 20
    }
  }
}

If you are staying on legacy cluster mode temporarily, omit cluster_hash_tag.

TLS / SSL

{
  "database": {
    "module": "hivemind-redis-db-plugin",
    "hivemind-redis-db-plugin": {
      "name": "clients",
      "host": "redis.example.com",
      "port": 6380,
      "use_ssl": true,
      "ssl_certfile": "/path/to/client.crt",
      "ssl_keyfile": "/path/to/client.key",
      "ssl_ca_certs": "/path/to/ca.crt",
      "ssl_cert_reqs": "required",
      "ssl_check_hostname": true
    }
  }
}

ssl still works as a backward-compatible alias, but use_ssl is the preferred config key in new docs and new deployments.

If your HiveMind-core config already includes subfolder, you can leave it there. This backend accepts it, but does not use it for Redis namespacing.

Configuration Reference

Runtime Behavior

• search_by_value("name", value) and search_by_value("api_key", value) use RediSearch when available.
• If RediSearch is not available, the backend falls back to Redis set indexes.
• In legacy Redis Cluster mode without cluster_hash_tag, the backend treats primary client records as authoritative and falls back to scan-based search for correctness.
• Search stays exact-match. RediSearch works as an accelerator, not as fuzzy search.
• Revoked clients are excluded from iteration and search results, so HiveMind admin flows only see active clients.
• sync() repairs drift by rebuilding counters, Redis set indexes, and RediSearch hash documents from stored client records.

Python Usage

from hivemind_plugin_manager.database import Client
from hivemind_redis_database import RedisDB

db = RedisDB(host="127.0.0.1", port=6379)

assert db.health_check()

client = Client(client_id=1, api_key="alpha-key", name="alpha")
db.add_item(client)

matches = db.search_by_value("name", "alpha")
print([c.name for c in matches])

client.name = "beta"
client.api_key = "beta-key"
db.update_client(client)

db.remove_client(str(client.client_id))
db.sync()

Cluster example:

from hivemind_redis_database import RedisDB

db = RedisDB(
    cluster_nodes=[
        {"host": "redis-node1", "port": 6379},
        {"host": "redis-node2", "port": 6379},
    ],
    cluster_hash_tag="clients",
    password="your_password",
)

Migrating Existing Redis Cluster Deployments

Do not enable cluster_hash_tag in place on an existing cluster namespace. That changes the Redis key layout.

Safe rollout:

1. Stop HiveMind writers for a short maintenance window.
2. Back up the current Redis namespace.
3. Run a dry run to confirm the source and target namespaces.
4. Run the migration tool to copy records into the tagged namespace.
5. Update server.json to set cluster_hash_tag.
6. Restart HiveMind and test the add, get, update, revoke, and search flows.
7. Keep the legacy namespace for rollback until the new deployment is stable.

Dry run:

hivemind-redis-migrate-cluster \
  --config ~/.config/hivemind-core/server.json \
  --target-cluster-hash-tag clients \
  --dry-run

Migration:

hivemind-redis-migrate-cluster \
  --config ~/.config/hivemind-core/server.json \
  --target-cluster-hash-tag clients \
  --clear-target

Notes:

• --clear-target clears keys in the target namespace before copying
• --source-cluster-hash-tag lets you migrate from one tagged namespace to another
• the migration tool copies raw client records, then runs sync() on the target namespace

See docs/cluster_consistency.md for more detail.

Legacy Redis Cluster note:

• this mode is correctness-first rather than index-first
• primary client records are written directly
• name / api_key searches fall back to scans
• RediSearch acceleration is disabled
• cluster_hash_tag stays the recommended production mode for indexed, transactional cluster behavior

Where it fits

hivemind-core
  └── hivemind-plugin-manager  (DatabaseFactory loads plugins by entry-point)
        └── hivemind-redis-database  ← this repo
              └── redis-py (Redis / RedisCluster) + optional RediSearch module

The plugin registers under the hivemind.database entry-point group as hivemind-redis-db-plugin.

Related Projects

• hivemind-core: the HiveMind server that loads this plugin
• hivemind-plugin-manager: defines the AbstractDB contract this plugin implements
• hivemind-sqlite-database: a sibling database plugin, and the source for the plugin-authoring guide

Docs

• docs/architecture.md — key schema, index design, RediSearch, sync()
• docs/configuration.md — full config reference
• docs/cluster_consistency.md — cluster hash tag, migration plan
• docs/operations.md — backup, authoring a plugin

Development

Useful local checks:

python -m unittest discover -s tests -v
python -m build --wheel

CI covers:

• unit tests
• single Redis Stack integration
• TLS Redis integration
• Redis Cluster integration
• Redis Stack Cluster integration

License

See LICENSE.