DefStream-Redis

CI and PyPI releases

The CI workflow tests PRs targeting master and pushes to master on Python 3.9 through 3.14. It builds all three DefStream packages from source, installs their wheels in a clean environment, and runs this repository's tests with RabbitMQ 4.1 and Redis 7.4 service containers. The current repository uses the event commit; sibling repositories use their master branches, with resolved commit IDs recorded in the log. All three GitHub repositories must be accessible. This bootstrap setup does not require existing PyPI releases and does not certify compatibility with previously published dependencies.

There is no VERSION or pyproject.toml file. setup.py configures setuptools-scm via setup_requires; python -m build resolves the build dependency in isolation. This legacy declaration may emit a deprecation warning. setuptools-scm derives versions from Git tags. CI fetches full history and creates a local candidate tag <RELEASE_SERIES>.<PATCH>.<github.run_number> (for example, 0.1.3.123) on every master push. Source changes increment PATCH by one; all other changes keep MAJOR.MINOR.PATCH. BUILD is always the Actions run number: 0.1.3.123 -> 0.1.3.124 (docs) -> 0.1.4.125 (source). The highest matching version in pre-push history supplies the base, keeping reruns stable. The four-part value is the full public PyPI version. New tags have no v prefix; existing v tags remain readable. PRs only run tests using their Git-derived development version without a tag. The complete push range is compared, including additions, deletions, and renames. Changes only to tests, docs, samples, workflows, or setup.py increase only BUILD. PyPI files cannot be overwritten; every upload has a distinct version identifier. Change RELEASE_SERIES in the workflow to start a new major/minor series. No version commit or version-update PR is created; run numbers can have gaps.

After all tests pass, the publish job pushes the release tag to the tested commit before uploading the verified artifacts. A tag may exist if uploading subsequently fails; retry the run and check PyPI for publication status. Existing tags on other commits are never overwritten. Allow the workflow's GITHUB_TOKEN to write contents and permit numeric four-part tags in any tag rulesets.

After PyPI upload succeeds (or a rerun confirms the version is fully published), CI creates a GitHub Release titled and tagged with the same version, with automatically generated notes and the verified wheel and sdist attached. New releases remain drafts until both assets are uploaded. A retry reuses the release, replaces matching assets with the verified files, and publishes it. PyPI upload failures prevent this step. The existing write-enabled GITHUB_TOKEN is sufficient; no extra token secret is required. PyPI's project links include the GitHub Releases page for change history.

Fetch tags with git fetch --tags for local builds. Tagged commits produce release versions; later commits produce development versions, with local suffixes for dirty trees. Build from a Git checkout or a published sdist, not a GitHub source ZIP without Git metadata. Installed versions are available via importlib.metadata.version('DefStream-Redis'). Before migration, tag the exact source commits of existing PyPI releases in all three repositories, using their version numbers as tags. Untagged SCM dependencies may resolve below their required minimum version; initial release tags also provide the starting PATCH for numeric builds. A tag such as 0.1.3 must name the commit used to publish that version, not an arbitrary current commit. Adding that tag does not edit sources or upload anything to PyPI. A new release series needs a source change or an existing matching release tag before it can produce builds. Its first source release starts at PATCH 0. Build counter regressions fail explicitly. Legacy .postN tags remain readable; numeric builds sort above post releases of a three-part base.

Only after all six Python checks pass does a separate publish job upload the exact Python 3.13 build artifacts. A fully published version is skipped on a rerun; first-attempt collisions, incomplete releases, and unexpected network errors fail. Active runs are not cancelled; older pending runs may be superseded.

Configure PYPI_API_TOKEN as a repository Actions secret or a secret in the pypi environment. Restrict that environment to master; omit approval rules for fully automatic releases. Protect master with required PRs and all six Test Python 3.x checks, not the push-only Publish PyPI job. Never commit the token. Publish the core before this adapter so users can resolve its dependency.

Redis Streams broker adapter for DefStream, distributed separately from the core. Requires Python 3.9 or newer and Redis 6.2 or newer.

Installation

pip install DefStream-Redis

This installs DefStream>=0.1.0 and redis>=5.1.1. The distribution name is DefStream-Redis; the Python import path remains DefStream.Broker.Implements.Redis.

Usage

from DefStream.Broker.Implements.Redis import (
	Configuration,
	Connection,
)

broker = Connection(Configuration(
	host='127.0.0.1',
	port=6379,
))
broker.connect()
try:
	broker.create_queue('tasks')
	broker.queue('tasks').send({'message': 'hello'})
finally:
	broker.close()

A reachable Redis server is required. The adapter uses Redis Streams, not Pub/Sub or lists. It provides stream-backed queues, consumer groups, subscriptions, and error policies. Use its Connection with DefStream.Worker.Worker for worker execution.

ConnectionType.Pool is the default; ConnectionType.Cluster uses Redis Cluster and requires database 0. Exchanges and RPC are not supported.

Source layout

The contents of the former DefStream/src/Broker/Implements/Redis directory now live directly in this project's src directory. Packaging maps that directory to DefStream.Broker.Implements.Redis; this distribution does not own or overwrite the parent packages supplied by DefStream.

Development

With the core and both adapters checked out as sibling directories, run this from their common parent using your virtual environment's Python:

python -m pip install --config-settings editable_mode=strict -e .\DefStream
python -m pip install -e .\DefStream-RabbitMQ -e .\DefStream-Redis
python -m unittest discover -s .\DefStream-Redis\test -p "test_*.py"

The core uses strict editable mode to prevent the adapters' editable namespace entries from masking its packages. Keep the core's generated build link tree; reinstall the core after adding or removing its modules. Adapters use default editable mode.

Redis-specific tests live in this project's test directory, covering connections, queues, consumers, serialization, and broker integration. Broker examples live in sample. Shared Worker, Inspector, and cross-adapter contract tests remain in the DefStream repository and require both adapters.

Run python -m unittest discover -s test -p "test_*.py" from this project directory after installation. Integration tests are opt-in via DEFSTREAM_REDIS_TESTS=1; ordinary unit tests do not require a running broker or the RabbitMQ adapter. The integration suite exercises standalone Redis, not Redis Cluster.

Do not combine editable installs with a site-packages/DefStream link pointing directly at the core source: installing adapter files through that link can write into the core checkout. Remove such a link before installing the projects. Reinstall after changing package metadata or dependencies.

License

See LICENSE, copied unchanged from DefStream.