DefStream-Redis

CI and PyPI releases

The CI workflow tests PRs targeting master and pushes to master on Python 3.9 through 3.14. It builds all three DefStream packages from source, installs their wheels in a clean environment, and runs this repository's tests with RabbitMQ 4.1 and Redis 7.4 service containers. The current repository uses the event commit; sibling repositories use their master branches, with resolved commit IDs recorded in the log. All three GitHub repositories must be accessible. This bootstrap setup does not require existing PyPI releases and does not certify compatibility with previously published dependencies.

On a master push, the build uses major.minor.<github.run_number> from the checked-in VERSION (for example, 0.1.42). PR builds retain VERSION. No version commit, PR, or tag is created. Change major/minor to start a new series; run numbers can have gaps. Check for PyPI version collisions before replacing or renaming the workflow.

Only after all six Python checks pass does a separate publish job upload the exact Python 3.13 build artifacts. A fully published version is skipped on a rerun; first-attempt collisions, incomplete releases, and unexpected network errors fail. Active runs are not cancelled; older pending runs may be superseded.

Configure PYPI_API_TOKEN as a repository Actions secret or a secret in the pypi environment. Restrict that environment to master; omit approval rules for fully automatic releases. Protect master with required PRs and all six Test Python 3.x checks, not the push-only Publish PyPI job. Never commit the token. Publish the core before this adapter so users can resolve its dependency.

Redis Streams broker adapter for DefStream, distributed separately from the core. Requires Python 3.9 or newer and Redis 6.2 or newer.

Installation

pip install DefStream-Redis

This installs DefStream>=0.1.0 and redis>=5.1.1. The distribution name is DefStream-Redis; the Python import path remains DefStream.Broker.Implements.Redis.

Usage

from DefStream.Broker.Implements.Redis import (
	Configuration,
	Connection,
)

broker = Connection(Configuration(
	host='127.0.0.1',
	port=6379,
))
broker.connect()
try:
	broker.create_queue('tasks')
	broker.queue('tasks').send({'message': 'hello'})
finally:
	broker.close()

A reachable Redis server is required. The adapter uses Redis Streams, not Pub/Sub or lists. It provides stream-backed queues, consumer groups, subscriptions, and error policies. Use its Connection with DefStream.Worker.Worker for worker execution.

ConnectionType.Pool is the default; ConnectionType.Cluster uses Redis Cluster and requires database 0. Exchanges and RPC are not supported.

Source layout

The contents of the former DefStream/src/Broker/Implements/Redis directory now live directly in this project's src directory. Packaging maps that directory to DefStream.Broker.Implements.Redis; this distribution does not own or overwrite the parent packages supplied by DefStream.

Development

With the core and both adapters checked out as sibling directories, run this from their common parent using your virtual environment's Python:

python -m pip install --config-settings editable_mode=strict -e .\DefStream
python -m pip install -e .\DefStream-RabbitMQ -e .\DefStream-Redis
python -m unittest discover -s .\DefStream-Redis\test -p "test_*.py"

The core uses strict editable mode to prevent the adapters' editable namespace entries from masking its packages. Keep the core's generated build link tree; reinstall the core after adding or removing its modules. Adapters use default editable mode.

Redis-specific tests live in this project's test directory, covering connections, queues, consumers, serialization, and broker integration. Broker examples live in sample. Shared Worker, Inspector, and cross-adapter contract tests remain in the DefStream repository and require both adapters.

Run python -m unittest discover -s test -p "test_*.py" from this project directory after installation. Integration tests are opt-in via DEFSTREAM_REDIS_TESTS=1; ordinary unit tests do not require a running broker or the RabbitMQ adapter. The integration suite exercises standalone Redis, not Redis Cluster.

Do not combine editable installs with a site-packages/DefStream link pointing directly at the core source: installing adapter files through that link can write into the core checkout. Remove such a link before installing the projects. Reinstall after changing package metadata or dependencies.

License

See LICENSE, copied unchanged from DefStream.