Skip to main content

ddredis

pypi downloads versions codecov license

Redis helpers: generic JSON cache and distributed lock

Installation

Install the library using pip:

pip install ddredis

Development

The project runs entirely in Docker. Requires Docker and Fabric on the host:

fab build      # build the dev image
fab tests      # run pytest
fab linters    # run ruff (with --fix), ty and complexipy
fab shell      # IPython inside the container
fab bash       # bash inside the container

This project was generated from dd-lib-stub; run uvx copier update to pull in template updates.

Compatibility

Python 3.11 to 3.14. What changed in each release, breaking changes included, is in the GitHub releases.

Depends on redis>=6,<9 and ddutils>=0.4.0,<0.5.0. The tests run on fakeredis, no server is needed.

GenericCache

Caches domain objects in Redis as JSON. The domain class must provide model_validate_json and model_dump_json (pydantic models do), keys are prefixed with the snake-cased domain name, every entry gets the class ttl with ±10% jitter so a burst of writes does not expire at once, and Redis errors turn into "not found" instead of failing the caller.

The client is obtained for every call from client_factory, a ConnectionManagerFactory from ddutils called with alias, the way a ddsql adapter runs a query inside factory(alias=...): the factory owns which Redis the alias means, the scope the client lives in (one ScopedRegistry per alias, a client per process and event loop, since redis.asyncio connections bind to the loop they were opened in), its lazy creation and its close on shutdown.

from enum import Enum
from functools import partial

from ddutils.connections import ConnectionManagerFactory
from ddutils.scoped_registry import ScopedRegistry, process_loop_scope
from pydantic import BaseModel
from redis.asyncio import Redis

from ddredis.cache import GenericCache


class RedisConnectionAlias(str, Enum):
    CACHE = 'cache'


client_registries = {
    RedisConnectionAlias.CACHE: ScopedRegistry[Redis](
        create_func=partial(Redis.from_url, 'redis://localhost/0', decode_responses=True),
        scope_func=process_loop_scope,
        destructor_method_name='aclose',
    ),
}
redis_client_factory = ConnectionManagerFactory(client_registries, default=RedisConnectionAlias.CACHE)


class Profile(BaseModel):
    profile_id: int
    name: str


class ProfileCache(GenericCache[Profile]):
    ttl = 10 * 60  # seconds
    client_factory = redis_client_factory
    # alias = RedisConnectionAlias.CACHE  # None picks the factory's default


cache = ProfileCache()

await cache.create(key=1, value=Profile(profile_id=1, name='John'))   # SET :profile:1 ... EX ttl±10%
profile = await cache.get(key=1)                                       # Profile | None
await cache.update(key=1, value=profile)                               # alias for create
await cache.delete(key=1)

await cache.get_list(key_prefix='team-a')                              # every :profile:team-a* entry
await cache.delete_by_filters(key_prefix='team-a')

await redis_client_factory.close()                                     # on shutdown

A subclass binds the domain class (GenericCache[Profile]; TypeError for a bare or still generic base) and declares client_factory, a ConnectionManagerFactory (AttributeError when missing, TypeError when something else), both checked at class definition. alias defaults to the factory's default, ttl to 300 seconds; the jitter is ttl // 10, so a ttl under 10 seconds has none.

Keys are :<snake_case_domain>:<key>, with str(key) of whatever is passed; get_list(key_prefix) and delete_by_filters(key_prefix) run SCAN MATCH :<prefix>:<key_prefix>*, so key_prefix is a glob and the cost is the whole keyspace, not the matches. An entry that does not deserialize into the domain class reads as a miss and is skipped by get_list. The client may run with or without decode_responses.

Redis errors are swallowed on purpose: for a cache, "unavailable" and "miss" lead to the same fallback. Do not use the cache for state where None has a meaning of its own. Transient failures (a worker that was just reloaded, a dropped connection) belong to the client, not to the cache: redis-py 6+ retries the errors listed in retry_on_error with its default Retry (3 attempts, exponential backoff with jitter), so a production create_func looks like this:

from redis.asyncio import Redis
from redis.exceptions import ConnectionError, TimeoutError

create_client = partial(
    Redis.from_url,
    'redis://localhost/0',
    max_connections=20,
    socket_connect_timeout=5,
    socket_timeout=1,
    retry_on_error=[ConnectionError, TimeoutError],
    decode_responses=True,
)

RedisLock

RedisLock(client_factory, key, timeout=10.0, blocking=False, *, alias=None): a distributed lock on SET NX PX with a Lua release (redis-py's Lock, thread_local=False). The client comes from a ConnectionManagerFactory like the cache's, borrowed for every call: the object holds only the token that identifies this holder, and a new redis-py Lock is built around the client each time. key is the Redis key as given, no prefix. Acquiring a held lock raises AlreadyAcquiredError; with blocking=True it waits up to timeout and raises the same error when the wait runs out. The lock is not reentrant, but the object can be acquired again after release().

from ddredis.lock import AlreadyAcquiredError, RedisLock

try:
    async with RedisLock(redis_client_factory, 'sync-profiles', timeout=30):
        await sync_profiles()
except AlreadyAcquiredError:
    pass  # another worker is on it

timeout is the Redis-side TTL of the key, not a client-side cancellation: if the block runs longer, Redis drops the key and a competitor may enter while the first holder is still running. Pick a timeout that comfortably exceeds the worst-case runtime, or call extend() periodically as a heartbeat. extend() replaces the TTL with the full timeout counted from now rather than adding to the remaining time, so a long-lived holder beating steadily never pushes its expiry further out. release() tolerates a lock that already expired, extend() on one raises redis-py's LockNotOwnedError; either without a prior acquire() raises redis-py's LockError (both from redis.asyncio.lock, not re-exported).

RedisFeatureFlagStorage

The Redis backend of ddutils feature flags: RedisFeatureFlagStorage(client_factory, *, alias=None, key='feature_flags'), one hash under key (DEFAULT_FEATURE_FLAGS_KEY), the field is the flag name, the value its JSON record {"enabled": true, "proportion": 1.0, "payload": {}}. The client comes from a ConnectionManagerFactory like the cache's, with or without decode_responses; rename is one MULTI/EXEC pipeline. Redis errors raise FeatureFlagStorageError, so FeatureFlags keeps serving the last known values while Redis is down.

A value that is not JSON or not a valid record is logged with an ERROR entry and read as the defaults, which means off: a record broken by hand disables its flag. The parsing is the strict FeatureFlagRecord.from_dict, an extra key is as malformed as a wrong value, while enabled accepts what the record accepts (true, 1, "yes", ...). get_map keeps the valid records when one is malformed.

from ddredis.feature_flags import RedisFeatureFlagStorage
from ddutils.feature_flags import FeatureFlags

feature_flags = FeatureFlags(RedisFeatureFlagStorage(redis_client_factory))  # alias=..., key='feature_flags'

if await feature_flags.is_enabled('new_checkout'):            # default proportion: on or off for everyone
    ...
if await feature_flags.is_enabled('new_ranking', user_id):  # on for `proportion` of the users
    ...

Metadata

Release files for ddredis 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ddredis 0.3.0
File Size Uploaded Provenance Actions
ddredis-0.3.0.tar.gz 9.6 kB With provenance Details for ddredis-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ddredis 0.3.0
File Interpreter ABI Platform Actions
ddredis-0.3.0-py3-none-any.whl Python 3 none any Details for ddredis-0.3.0-py3-none-any.whl

Total release size: 21.4 kB

Release files / ddredis-0.3.0.tar.gz

Download URL ddredis-0.3.0.tar.gz
Size 9.6 kB
Tags Source
SHA-256 checksum
How to use checksums
fcf6f8b3367670f0fab613ee69f20536d77f19350afbce6ec58c56d7d191279a
BLAKE2b-256 checksum
How to use checksums
b3ae27c3fa83f02abb7123b5399cacc12c53b8f6ca4cb237c6e0685f36babb72
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 19, 2026.

Transparency log

Release files / ddredis-0.3.0-py3-none-any.whl

Download URL ddredis-0.3.0-py3-none-any.whl
Size 11.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a8ab046d80000aa791f05a1d757c035747c4813bcf923573d4ead8480a28da45
BLAKE2b-256 checksum
How to use checksums
112d00ee3f3f1198487704d6237dc93238e7960f86ea09271d077d86e1ab97a4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 19, 2026.

Transparency log

Security for ddredis 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for ddredis-0.3.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 19, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page