envs-xmpp

Shared technical infrastructure for the envs.net XMPP bots envsbot and muc_banbot.

One distribution intentionally ships two stable Python packages:

• envs_xmpp_core: bot-neutral runtime, XMPP, config, storage and release primitives.
• envs_xmpp_ops: deployment and operations primitives.

Keeping both in one distribution gives runtime and deployment infrastructure one version, one repository and one release pipeline while preserving the existing import APIs used by both bots.

The package contains no bot commands, permissions, database schemas, moderation logic, plugin systems or bot-specific lifecycle policy.

Compatibility

• Python 3.12, 3.13 and 3.14
• GPL-3.0-only
• no mandatory third-party runtime dependencies

The base package keeps no mandatory third-party runtime dependencies. Optional OMEMO support is available through envs-xmpp[omemo], while each bot still owns its encryption policy, commands, recipient-state adapters and operator wording.

Shared storage primitives include SQLite integrity checking and safe ZIP member validation/streaming. Pending room invites use a shared typed model, deduplication, store state machine and SQL repository while each bot keeps only a thin database-API adapter and bot-specific notification/join policy. Declarative config-schema primitives provide shared default/type/range/lifecycle metadata; each bot keeps its domain-specific validation, reload behavior and operator-facing wording local. Pagination provides a neutral page-slice model while bot frontends retain their existing command-specific return formats.

Shared config and restore operator contracts (development)

envs_xmpp_core.config.operator supplies deterministic config-diff previews, secret-redacted runtime change lines and structured reload diagnostics. A report is not validation or authorization: each bot must validate candidate values, keep startup-only settings inactive until restart, and use its existing locked config-file transaction and rollback. The reload renderer accepts only already-redacted diagnostic lines, never raw secret-bearing configuration.

envs_xmpp_core.storage.restore.restore_recovery_report() classifies an unsuccessful restore as a completed/incomplete rollback, completed/failed runtime recovery or no recovery attempt. It does not perform recovery and must not be treated as crash-atomicity of multi-file restoration. Application wording and post-restore restart policy remain bot-specific.

Shared command and help contracts (development)

envs_xmpp_core.commands contains metadata-only primitives shared by envsbot and muc_banbot. CommandSpec and SubcommandSpec describe help and usage without creating runnable bot commands. The shared resolve_structured_subcommand() function matches multiword commands and subcommand aliases with longest-match precedence; when two matches span the same input, the more specific registered command takes priority. Arguments retain their original casing, and placeholder names are never treated as literal invocations. resolve_help_topic() provides chained, nested topic alias normalization.

Security boundary: callers must filter commands and subcommands according to the requesting user's role before using the resolver. Actual dispatch, authorization, room policy and user-facing help presentation remain the responsibility of each bot. These development interfaces do not change the published package version until the planned joint release.

Shared room lifecycle (development)

envs_xmpp_core.runtime.RoomLifecycleRegistry records seven common MUC observations: configured, joining, joined, degraded, failed, deferred and leaving. It yields immutable RoomLifecycleSnapshot records, stores optional retry timestamps without choosing retry delays, and increments a session generation when a transport session is replaced. A new session always invalidates an earlier joined assertion, while explicit leave intent survives reconnect.

A room becomes joined only when its application adapter passes verified bot self-presence to confirm_self_presence(). This tracker is not an occupant authenticator or a second source of truth for privileges; both bots retain their existing verified occupant caches, join timers, reconnect policies, moderator rights and application-specific room configuration. These development interfaces will be released after the joint unification work, not through an interim version bump.

Installation

Install the stable package from PyPI:

python -m pip install envs-xmpp
python -c "import envs_xmpp_core; print(envs_xmpp_core.__version__)"

Install the optional OMEMO mechanism when a consumer needs encrypted transport:

python -m pip install 'envs-xmpp[omemo]'

For development, install a checkout with the repository quality and mutation tools:

cd /path/to/envs-xmpp
python -m pip install -e ".[dev]"

A plain pip install -e . installs only the runtime package. The dev extra is required for ./scripts/quality.sh and mutation testing.

Existing imports remain valid:

from envs_xmpp_core.runtime.tasks import TaskSupervisor
from envs_xmpp_ops.profile import DeploymentProfile

Package layout

src/
├── envs_xmpp_core/
│   ├── config/
│   ├── pagination.py
│   ├── presentation.py
│   ├── release/
│   ├── runtime/
│   │   ├── alerts.py
│   │   ├── diagnostics.py
│   │   ├── health.py
│   │   ├── reconnect.py
│   │   ├── rooms.py
│   │   └── session.py
│   ├── security/
│   │   └── redaction.py
│   ├── storage/
│   │   ├── archive.py
│   │   ├── files.py
│   │   ├── outbox.py
│   │   └── sqlite.py
│   └── xmpp/
│       ├── affiliations.py
│       ├── avatar.py
│       ├── jid.py
│       ├── messaging.py
│       ├── muc_join.py
│       ├── occupants.py
│       └── omemo/
└── envs_xmpp_ops/
    ├── accounts.py
    ├── deploy.py
    ├── git.py
    ├── interaction.py
    ├── layout.py
    ├── paths.py
    ├── profile.py
    ├── release.py
    ├── release_audit.py
    ├── service.py
    ├── systemd.py
    └── venv.py

Stable 1.x API

Version 1.0 formalized package-level convenience imports for the shared infrastructure introduced during the bot consolidation. The public surfaces are envs_xmpp_core.xmpp, envs_xmpp_core.storage, envs_xmpp_core.runtime, envs_xmpp_core.security, and envs_xmpp_ops. Direct module imports remain supported, so existing consumers do not have to migrate immediately.

The stable shared layer now covers avatar/profile publication, confirmed MUC joins, normalized occupant identity, message-target/reply routing, durable outbox storage, operational alert state, redacted diagnostics and deployment layout discovery. Version 1.1 adds the shared operator-presentation models and renderers for task, status and room inventories. Version 1.2 adds shared JID text normalization, including deliberately narrow cleanup of U+200B/U+FEFF copy/paste artifacts before best-effort bare-JID comparison. Version 1.3 adds shared reconnect retry/backoff orchestration that waits for full application readiness, avoids duplicate transport attempts during session-start races, and resets partial sessions after bounded startup timeouts. Version 1.4 adds declarative release-tag and wheel verification for consumer repositories, including packaged-asset integrity, console entry-point validation, isolated installation, pip check, runtime asset resolution and CLI version smoke tests. Version 1.5 adds shared coverage- and mutation-regression gates, including reviewed survivor baselines and strict rejection of new survivors or incomplete mutation outcomes. Version 1.6 adds shared optional OMEMO mechanism primitives: private storage and identity rotation, XEP-0384 plugin/storage integration, fail-closed decrypt/encrypt helpers, unusable-recipient filtering, conservative device hints and task-local reply encryption state. Version 1.6.1 makes the shared-core release audit extras-aware (for example envs-xmpp[omemo]>=...) and includes Python 3.14 constraint alignment. Version 1.7 expands the shared command, messaging, task, room lifecycle, configuration and backup infrastructure used by both bots. Version 1.7.1 centralizes the reviewed dependency-drift deployment gate, and 1.7.2 adds DeploymentFrontend to bind the remaining common process, Git, systemd, virtualenv, confirmation and service-account deployment operations. The bots still own encryption policy and commands. The bots still own disconnect cleanup, room/state reconciliation, readiness criteria, alerts and other application policy. The 1.x line also shares XMPP session-generation telemetry and bounded MUC affiliation IQ mechanics while keeping strict JID validation and bot policy in the applications. Bot-specific command behavior, moderation, OMEMO policy/commands, plugin systems and notification wording intentionally remain outside the core; only the reusable OMEMO mechanism is shared.

envs_xmpp_ops is designed for thin bot-specific deployment frontends. The frontends subclass the shared DeploymentTarget for common checkout/venv/config/ service coordinates and keep bot policy such as config migration, database backup/restore and service hardening local. Shared Git release selection, systemd inspection, operator confirmation, account/path checks, virtualenv creation, consumer release-state auditing and generic release artifact verification live here.

Fresh installs do not assume that this package is already present. Each bot ships a tiny stdlib-only bootstrap shim. When the exact required envs-xmpp version is unavailable, that shim creates a versioned cached deployment virtualenv below $XDG_CACHE_HOME/envs-xmpp/deploy/ (or ~/.cache/envs-xmpp/deploy/), installs the pinned version from PyPI, and re-executes the deployment frontend. ENVS_XMPP_DEPLOY_SOURCE can point at a local checkout or wheel for development and pre-release testing.

Documentation

• Developer guide
• Architecture and ownership boundaries
• Stable API reference
• Consolidation policy

The documentation intentionally distinguishes stable public imports from internal implementation details. Bot-specific policy should stay in the applications unless it has demonstrably identical semantics in more than one consumer.

CI and PyPI releases

GitHub Actions tests Python 3.12, 3.13 and 3.14. A vX.Y.Z tag is accepted only when it exactly matches project.version in pyproject.toml. Release distributions are published through PyPI Trusted Publishing/OIDC, without a long-lived PyPI token.

PyPI publishing is configured through the pypi GitHub environment and the Trusted Publisher for envs-net/envs-xmpp using .github/workflows/release.yml.

Shared developer quality runners

envs_xmpp_ops.quality and envs_xmpp_ops.testing provide the common local quality/test frontends used by envsbot and muc_banbot. Project-specific source targets, generated-file checks, integration markers, coverage floors and regression baseline paths remain declarative in each repository's pyproject.toml; the runner behavior and gate ordering stay shared.

The quality runner enforces a common baseline: compilation, project validation, warning-strict tests with coverage and an accepted coverage-delta gate, Ruff repository/F401/I-UP-B gates, mypy, Git whitespace validation and dependency audit. envs_xmpp_ops.regression also provides the shared mutmut survivor-delta gate used by the repositories: known survivors may be accepted, while new survivors and all no tests, timeout, suspicious, or incomplete outcomes fail. Consumer project validation can include python -m envs_xmpp_ops.release_audit, which verifies that package metadata, requirements, constraints and the deploy bootstrap agree on one shared-core version before a release is cut.

Runtime and utility primitives

The shared core also provides heartbeat-aware worker waits, lifecycle phase orchestration, passive HealthCheck/HealthSnapshot diagnostics with failure-isolated collection, normalized task/watchdog/lifecycle/room-inventory facts, ordered health-message flattening, asynchronous release comparison results, and neutral human-readable duration/byte formatting. Applications keep notification policy, active recovery behavior, severity decisions, concrete domain checks and labels locally.

Shared incoming message context (development)

envs_xmpp_core.xmpp.message_context_from_stanza() creates an immutable MessageContext snapshot of incoming XMPP routing metadata (DM, MUC or MUC-PM). Call it after decryption to read the command body and encryption flag. The reply_route describes the wire destination/type; it is not an OMEMO recipient or an authorization decision. An occupant's real JID is recorded only when the application provides one explicitly. XEP-0359 origin-id and the ordinary stanza id are kept separate. This helper does not persist decrypted messages or change either bot's encryption, caching or admin policy.

Shared reply and outbound delivery planning

envs_xmpp_core.xmpp.outbound provides plan_outbound_message, resolve_reply_encryption, can_persist_without_encryption_context, and ensure_message_origin_id. These are transport-neutral contracts: an explicit OMEMO reply is not eligible for plaintext outbox persistence, and the same persisted XEP-0359 origin-id should be reused on every replay, including newly produced encrypted wire stanzas (encrypt_and_send(..., origin_id=...)). The application still owns encryption recipients, opt-in fallback policy, transport readiness, and its own queue-first or send-first retry strategy.