Native Hermes adapter distribution

Hermes 0.21.1 applies hooks.output_spill to external memory-provider output. Its default 10,000-character head/tail preview can cut a Colony source envelope or remove the middle of an original/correction bundle. Guided attachment and explicit colony init --refresh-adapter align hooks.output_spill.max_chars to 65,536 characters, retaining an existing larger cap or disabled spill setting. The original configuration is backed up and an adjustment is printed during refresh. The managed draft profile receives the same alignment at creation, role refresh and adapter refresh. Restart the selected Hermes gateway to load this provider setting; newly spawned workers load their own profile setting.

This allowance leaves headroom above the selected-memory ceiling of 24,000 characters for citations and other default context sections. It does not increase retrieval budgets, shorten corrections or change model context capacity. This is a shared Hermes hook setting: unusually large custom context producers must size their own transfer allowance. The aggregate of optional context sections has no universal size bound. A native MemoryManager regression verifies a complete envelope larger than the old default survives the installed configuration unchanged.

colony-hermes packages the existing general adapter and memory provider for installation into the Python environment that runs Hermes. The Colony sidecar is a separate service. The adapter does not install the sidecar, a context engine, a worker daemon, or operating-system services.

Build from the repository root:

python -m pip install build
python -m build

Install the resulting wheel with the Python interpreter that runs Hermes:

python -m pip install dist/colony_hermes-1.0.0-py3-none-any.whl

The wheel exposes colony through hermes_agent.plugins and colony-memory through hermes_agent.memory_providers. It maps the canonical source files in plugins/hermes-plugin/ and plugins/colony-memory/ to importable packages. Only catalog.py and contract.py from hostworker/colony_hostworker/ are included in the adapter's private catalog package. Source-checkout plugin paths and the existing installer's copying behavior remain compatible.

Activation and current limits

Installing the wheel makes the adapters discoverable. It does not change a Hermes profile, select a memory provider, or enable tools. Activation requires the existing general-adapter configuration, plugins.enabled: [colony], and memory.provider: colony-memory in the selected private profile. Preserve other enabled plugins when editing that list. These two durable selections make the general plugin the canonical writer and keep the memory provider read-only, including cold native workers that do not inherit launcher environment flags. An explicit plugins.disabled: [colony] or an enabled list excluding colony takes precedence over inherited flags. A contradictory configured provider turn_writer: enabled is rejected; native colony-memory.json settings retain their precedence over legacy memory.config.

Older embedded profiles without this paired selection still require the existing coexistence settings in the Hermes process environment:

COLONY_GENERAL_PLUGIN_ACTIVE=1
COLONY_MEMORY_WORKER_TOOLS=0
COLONY_MEMORY_TURN_WRITER=disabled

Configure the sidecar URL and contact through native hermes memory setup and matching plugins.colony configuration, and supply COLONY_API_KEY privately. Native setup stores non-secret fields in the selected profile's colony-memory.json, which overrides legacy memory.config. The general adapter needs a private writable turn outbox and verified participant bindings. Consequential tools retain their existing mediator requirements. This packaging change does not provision those dependencies.

Do not silently replace a selected external memory provider. Existing directory installations also need explicit migration: Hermes gives directory memory providers precedence over pip providers, while a pip general plugin can override a same-name directory plugin. Remove or archive obsolete plugin directories only as part of an intentional profile migration.

When the memory provider is selected, native CLI discovery exposes hermes colony-memory status, goals, context, and sync. These commands resolve the same selected profile settings and credentials as the provider; explicit URL/contact arguments remain available. The Typer app remains available to existing callers.

Profile settings and handoff files stay scoped to the selected Hermes home. The provider remains attached through a sidecar startup outage and retries on later requests. Automatic profile activation remains follow-up work; packaging alone does not establish production readiness.

Durable source capture

The native memory provider implements compression checkpoint API v2. Set compression.checkpoint_required: true to make Hermes retain its transcript when local checkpoint persistence fails. A successful checkpoint commits the normalized direct source messages into the same private SQLite outbox used by the general adapter. The callback then attempts delivery within 250 ms. Sidecar downtime leaves a pending durable record and permits compression; pending does not mean centrally recallable. Diagnostics expose checkpoint state. The ordinary turn writer and subsequent checkpoints drain that outbox.

The outbox accepts at most 8 MiB of serialized data per record. Oversized or unserializable evidence fails explicitly without clipping it. System rows, tool wrappers, compression summaries and injected api_content are excluded. Direct message content, including media references, is retained. The general turn writer also retains complete messages instead of cutting at 2,000 characters. New records require a source_recorded receipt before delivery is acknowledged, so an older sidecar cannot silently discard an unfamiliar checkpoint payload during an upgrade.

The existing /v2/host/turns/{turn_id} endpoint stores direct source JSON and a lexical index transactionally in turn-idempotency.db. Checkpoints bypass ordinary interaction, affect, graph and initiative effects. Replays do not create another source. occurred_at is retained when supplied explicitly in context metadata; otherwise it is unknown. Server ingested_at is separate and is never treated as the date asserted by the conversation.

/v1/host/context/assemble reads the source index after the existing exact viewer check. Ordinary attributed turns can be recalled across that person's sessions. Full-history checkpoints additionally require their original session, because the native history does not guarantee old per-message speaker attribution. Media references remain source data; only text is indexed. This is direct evidence recall, not an embedding migration or a belief contradiction engine.

Source erasure and replay

POST /v1/host/memory/sources/forget accepts an authenticated contact and 1 to 100 canonical source IDs. The existing MCP server exposes colony_forget_sources. The native plugin exposes colony_memory_forget for explicit owner requests in an attested interactive turn. It accepts source IDs from canonical recalled provenance, including older sessions; legacy graph memory IDs are not source IDs. Native Hermes committed memory removes also attempt an exact, session-bound old_text match, including when the general plugin owns ordinary turn writes. No match or multiple matches produces an explicit unmapped/ambiguous diagnostic; it never broadens a text search into deletion. IDs unknown to the central source store are rejected. Pending-only local evidence needs to be identified at its host.

Erasure commits source-ID and exact-message hashes before cleaning linked graph summaries. Checkpoint copies are redacted within the same contact and session; unrelated messages remain. New graph summaries carry source_uri=turn:<id> and source_turn_id. Projection markers block late writes and reads while deletion is pending. The response separates source erasure from graph cleanup and host reconciliation. A repeated request retries the same derived cleanup targets.

New native answers retain the authorized canonical source revisions supplied in their recalled context. Removing one revision also removes recorded dependent assistant messages, including chains recalled in later sessions. This records conservative input dependency, not proof each source influenced every word: an entire assistant message can disappear even if it also contains other useful material. Independent user messages remain. After partial erasure their new source revision can support future answers. Generated citations and user-typed context markers cannot establish these dependencies; historical unlinked paraphrases are not reconstructed. Other transports need to forward the same structured references before their answer copies have this guarantee.

The host outbox migrates its existing v1 database transactionally to v2 with a separate contact-bound erasure watermark. Canonical turn/checkpoint delivery fetches /v1/host/memory/sources/erasures before PUT. A missing endpoint, outage, incomplete page or server history behind the host cursor holds replay. The host purges both pending payloads and delivered receipts, preserving unrelated pending messages as evidence-only checkpoints. Authenticated ordinary survivors retain their sender and person scope through a source-only route, which schedules user claim projection without replaying summary/tool/relationship effects. A predecessor server rejects that dedicated route, so its host must keep the survivor queued. Ordinary answers carrying source references also use a dedicated route, preventing a predecessor backend from silently accepting an answer while discarding its links. Erased IDs cannot be enqueued again after reconciliation. This is not a model-generation counter. Generic caller-provided outbox delivery callbacks must use ColonyClient.sync_turn(..., outbox=outbox) to participate in reconciliation.

Partial erasure events retain exact message hashes under opaque event IDs in the existing cursor sequence. Predecessor readers still filter those message copies without treating surviving user sources as wholly deleted. Downgrading preserves completed erasures, but predecessor writers do not record new answer dependencies.

This is scoped source erasure, not a claim of global forgetting. Native Hermes transcripts/API context, backups, prior graph records without source lineage, legacy shared facts, ToM, commitments, and other old derivative stores need their own erasure adapters. Offline hosts retain bytes until reconnecting; filesystem snapshots and physical-media remnants are outside this logical-delete contract. Erasure history must survive backup restores; replay detects an older watermark but cannot prove that a replaced server with reused sequence numbers is equivalent.

Qualification

Pull-request CI keeps an exact qualified Hermes commit pinned. The separate Latest Hermes compatibility workflow runs daily and can be dispatched manually. It selects the latest published stable upstream release, records its exact Git commit and runs the same built-artifact/native-profile tests. It uses disposable profiles and controlled inference, without deployment credentials or a live agent. A compatibility failure leaves production and the qualified pin intact.

When a new upstream release passes, update the pin in an ordinary reviewed change and qualify the deployment's actual channels and recovery before moving that deployment. The scheduled result catches upstream drift; it does not establish production readiness or automatically upgrade a running agent.

The qualification target is Hermes v0.21.1, tag v2026.9.7, commit 2237be355906fbe6065ce1815711eee52b2d646e, tested on Python 3.12. Hermes 0.21.0 attachment remains supported. The adapter uses canonical 0.21.1 task and skill modules, with a narrow fallback when those modules are absent on 0.21.0. It does not depend on upstream's temporary deprecated-import shims. The package allows Python 3.11 through 3.13; those other interpreters are not yet qualified. Other Hermes releases are unqualified until the native-loader checks pass against them. Hermes is installed separately; this package does not select or upgrade it.

Hermes 0.21.1 wraps provider recollection in a note calling it authoritative reference data. Colony's supported llm_request middleware replaces that outer note only in an observed, current, source-stamped native memory suffix. It keeps the person’s input, quoted source bytes, source revisions and erasure checks unchanged. Recollection retains its speaker, time and factual, reported, fictional or hypothetical scope; finding a quotation does not verify its claim. The shared memory section header carries this guidance to direct consumers such as voice adapters that do not use the Hermes envelope. Unknown envelope formats are unchanged. The native request test exercises the actual upstream wrapper, so a framing change requires renewed qualification rather than a core patch.

System/developer instructions and Responses instructions may document the generic <memory-context> fence. That markup alone is not a recalled packet there, even when the example omits a closing tag. Exact forgotten source copies and explicit Colony lineage packets still reconcile in these fields. Legacy untagged fenced instruction text is preserved unless it is an exact erased source or observed alias; substring and paraphrase erasure are not promised. Native automatic recollection continues to reconcile in its appended user-content boundary.

Install test dependencies into an isolated environment with the target Hermes checkout available, then run:

python -m pip install build pytest "setuptools>=77" wheel
python -m pytest tests/hermes_adapter -q

The tests build a wheel and a source distribution, rebuild from the source distribution, install the wheel outside the checkout, and exercise the actual Hermes general and memory loaders. Native-loader tests require Hermes and are explicitly skipped when it is absent. No live sidecar, model endpoint, production profile, or channel is contacted.

Shared execution observations

On the supported Hermes 0.21.0 and 0.21.1 releases, set plugins.colony.execution_registry_enabled: true to publish native turn, API, tool and delegated-child lifecycle observations. This is opt-in for existing installations. The adapter credential must already have scoped turns:write and exact person grants; context:read grants access to the corresponding view. The legacy global bearer does not attest identities for this new surface. To observe trusted agent cron fires, include cron in the existing attested_system_platforms configuration; it is not enabled by the adapter's default cli binding. No network call is made during registration.

Colony stores only execution IDs, participant/session linkage, channel, phase, tool name and observation times in the existing turn-idempotency.db. It does not copy prompts, tool arguments/results or task descriptions. Each hook has a 400 ms network deadline and failures do not stop a turn. Ordinary use supplies new observations at API/tool boundaries; there is no heartbeat thread. After 120 seconds without an observation, liveness is unknown, including during a long model response. A terminal native turn event closes its observation; late events cannot reopen it. Metadata older than seven days is hidden from the view and removed during subsequent observation writes.

GET /v1/host/executions?contact_id=<bound-person>&session_id=<session> returns the scoped view with age, coverage and truncation fields. The owner, identified by server configuration and an existing exact person grant, can see registered turns across sessions. Other people can see only their own selected session; public/guest turns receive no automatic registry injection. The owner's normal context assembly includes up to eight observations when present.

The native subagent_start event binds each executing child to the exact observed parent turn. Child observations inherit that participant and cannot broaden it. Observation does not grant tool authority; the separate enforcement below also works when observation is disabled. No model argument selects the writer or owner role.

The view observes running Hermes turns, including agent cron fires and executing subagents, and reads claimed/running workers from the canonical task queue for the owner. It also reads native cron attempts, including script-only jobs, from one explicitly bound Hermes profile. The sidecar uses its private local-instance manifest or explicit HERMES_HOME; it never scans other profiles or assumes a default home. A conflicting binding is unavailable. The existing CLI-managed service supplies the same private home automatically.

The cron projection reads cron/executions.db without importing or running the native scheduler, initializing its schema, or rewriting its status. It returns native execution/job IDs, a hashed source-home identity, optional job names, record timestamps and claimed/running status, plus a bounded day of recent completed/failed/unknown records. Prompts, scripts, output, errors and credentials are excluded. No observation writer or polling daemon is added. Native recorded running state has unknown process liveness, however recent it is. Only Hermes can reconcile abandoned attempts; completed records do not prove a third-party effect. Guests receive no native cron rows. Missing, incompatible or unreadable ledgers are explicitly unavailable, and complete is always false for the combined work view.

The same owner view reads native Kanban tasks, including general goal_mode work, without dispatching or reconciling them. COLONY_HERMES_WORK_BOARDS can select up to eight board slugs as a JSON list, for example ["default","colony-drafts"]. Without that setting it follows only the selected home's native current board (HERMES_KANBAN_BOARD, then kanban/current, then default), including Hermes' lowercase normalization. It does not enumerate other boards. Profile homes use Hermes' shared root for board paths; conflicting explicit native home/database overrides are unavailable.

Each task includes a title capped at 128 characters, quoted as operational data, native task/run IDs and states, configured goal mode/turn budget and record timestamps. Bodies, arbitrary result prose, claim tokens and worker PIDs are excluded. The per-request native work refresh includes these titles and states, so another owner session can identify an undertaking and observe its later terminal record. Goal budgets describe configured limits, not remaining turns or evidence of completion. Running rows retain unknown process liveness; terminal rows do not prove an external effect. Missing selected boards, partial board coverage and omitted rows remain visible. Recent terminal records are bounded to seven days using their actual terminal timestamp. Archiving unfinished work uses its native archive event for terminal_record_at and leaves completed_at unset. Guest context receives no Kanban board rows. Hermes retains task creation, goal continuation, completion, recovery and all execution authority.

New dispatcher-owned completions can also retain their finalized native run summary as explicitly machine-authored, unverified assistant evidence. The kanban_task_completed hook re-reads the completed native run while the actual kanban_complete tool still holds its attested CLI participant context. It copies the canonical source revisions actually supplied to that request, without using generated citations. Task bodies, worker instructions, continuations and artifact files are not forwarded. A missing scope or request-lineage observation leaves the report native-only and records a structural log reason; old completions are not backfilled. This separately scoped assistant-report hook does not depend on which platforms enable ordinary conversation capture; excluding CLI user turns still retains eligible native completion reports. An unrelated dispatcher-owned board worker omits the profile's draft-board controller and rejects local-draft acceptance before any handoff. Native's worker DB pin stays intact; the selected draft worker still requires its own board and profile. General scope, request lineage and completion hooks remain available in the ordinary worker.

The existing outbox and source-only route retain this assistant summary for scoped lexical and semantic recall after the seven-day operational window. They do not run ordinary turn cognition or promote it into a user's claims. Repeated callbacks reuse one native-home/board/task/run identity, including after a lost acknowledgement. Erasing a supplied parent removes the dependent report; native task and session history remain under their existing retention policy. Queued delivery is not proof of central availability, and a reported result is not independent verification that its claimed effects occurred.

An optional private COLONY_WORKER_STATUS_PATHS environment value maps neutral worker labels to local JSON heartbeat paths, for example {"Local transport":"/private/runtime/transport-heartbeat.json"}. It is unset by default. The same owner API and context join expose these as separate reported_worker entries, never executions or success receipts. Heartbeats supply state and numeric Unix updated_at; optional detail_code and release_commit are included. Existing heartbeats remain compatible. The reader also retains optional task_id, parent_task_id, worker_id, kind, started_at, finished_at, and integer exit_code. A parent ID is a reported binding, not permission to act for that parent. Each row includes status_sha256 to identify the exact observed report bytes. Configured status paths, process arguments, environment values, logs and arbitrary result bodies are omitted.

Producers can include up to four measured progress records: {"completed":3,"total":8,"unit":"files","source":"pinned_manifest_metadata","observed_at":1234}. total is optional; the other fields are required. Counts must be nonnegative and finite, and a supplied total cannot be less than the count. The source and measurement time must describe the actual observation. Reading artifact progress must not refresh an old process heartbeat or imply an unmeasured transfer rate.

Up to four result_refs can contain kind, an opaque reference, and optional SHA256 sha256, numeric observed_at, and a short verification description. For example, a reference can identify a retained process exit receipt or artifact report. The reader never follows those references or verifies the referenced content; it reports the producer's stated scope of checking. Explicit result locations therefore belong only in operator-selected private reports. Automatic request context retains these same bounded work fields and references, subject to its existing overall context limit. The complete scoped API remains available.

The reader reports age and marks reports older than 120 seconds stale; future timestamps and unreadable/malformed files remain unknown or unavailable. States such as uncertain are preserved even when stale. A task with a valid finish timestamp and explicit exited, completed, failed, interrupted or cancelled state is labelled terminal_report; otherwise it is a progress_report. An old terminal report remains intelligible after a session disconnect. A stale running report or missing file never becomes completion. Process liveness and external effects remain unverified. Reads are limited to eight configured workers and 16 KiB per heartbeat; no store, heartbeat writer or poller is added. Guests receive no configured worker reports.

Qualification fires a due native no_agent job in a disposable profile with deliver=local, observes the same execution ID during and after actual script completion, and checks the persisted output. This does not depend on an empty enabled_toolsets, which Hermes treats as its default tool selection. A separate installed-wheel test runs actual native parent and child turns with controlled inference, reads a neutral local file, delivers the asynchronous completion through the native CLI ownership filter, and resumes the parent. A foreign session cannot consume that completion; a guest child retains guest tool limits. These tests do not contact a model endpoint or a production profile.

The view still does not cover queued children, unregistered external coding processes or every hardware service. It does not enforce atomic free-text conversational commitments. The optional explicit shared undertaking coordinates work against an existing person commitment ID in its durable store. Seeing concurrent work is a prerequisite for coordination, not a guarantee that promises cannot conflict.

Native tool authority

On the qualified Hermes release, the general plugin enforces participant authority at native tool_execution middleware. Resolved owner turns and explicitly configured local system platforms retain native tools and existing Hermes toolset, approval and guardrail checks. Guests cannot directly invoke shell, files, network, devices, coding, delegation, native memory/session tools, or unknown plugin/MCP tools. The exact Colony tools keep their existing scoped read checks and action mediators. No arbitrary tool prefix or relationship score grants access.

A native-tool request from a guest returns requires_authorization, with no effect and no approval created. The response directs the agent to an enabled Colony action request or the owner. It does not invent an approval request or permit a raw tool after conversational consent. Broader guest capabilities require a scoped mediated interface; this packet supplies none automatically.

Authority uses the transport-resolved participant bound to the exact native session/task/turn, never fields in model arguments. A native child-spawn event binds the child to the exact parent's authority. Missing or conflicting parent bindings cannot acquire the local CLI system role. Native pre-API callbacks retain that binding across compression session rotation. Coding RPC calls use Hermes' propagated execution context, without a process-global last-owner or task-only lookup. Unavailable identity returns an explicit error because Hermes middleware exceptions otherwise fail open. Local trusted CLI remains available when contact resolution is offline; remote identity is not guessed.

This is a boundary for model-requested tool dispatch while the general plugin is loaded. It does not sandbox trusted installed plugin code, remove private facts already present in old transcripts/system prompts, or replace the deployment's channel admission, secret isolation and consent policy. Native owner tools retain their existing consent behavior; they are not converted to Colony intents by this gate. Qualify both owner and guest messages before enabling a public channel.

Native post-turn review

Hermes' automatic review reuses its parent's session ID without emitting a subagent-start event. The adapter captures the exact parent participant on the native caller thread and accepts inheritance only with Hermes' trusted review provenance, matching parent/session, and a distinct task/turn. Later speakers in the same session cannot upgrade a queued guest review. Reviews get a separate work record linked to their parent, do not replace the session's current speaker, and do not write their internal harness into ordinary conversation evidence.

For owner/system reviews, skill_manage produces a proposal in Hermes' existing pending/skills store. Native curator ownership checks still apply. Foreground owner requests retain native behavior. The proposal is not an evaluated or activated improvement: an explicit operator or a separately configured task evaluator must use the native apply path. Mutation history and rollback remain in Hermes' skill ledger. No new consent transport or learning store is added. Batch proposals require an actual nonempty array of at most 20 operation objects with supported actions and target names; deletion must be the sole operation. Encoded or malformed strings are rejected before staging; the adapter does not repair them silently. Valid native legacy flat operations remain supported. Existing pending records are left intact.

Installed-wheel qualification runs real native automatic review threads with controlled inference for owner and guest, including a later-owner race in the guest's session. It checks proposal persistence, exact inherited authority, separate work observation, absent harness capture, and explicit native apply/ledger rollback. It does not claim measured task improvement from that controlled model fixture.

Measured skill updates

python -m colony_hermes.review_evaluation --skill NAME --pending ID --oracle trusted_local_module:function evaluates one explicitly selected, existing curator-owned SKILL.md proposal. It accepts native full-content edits or exact text patches, including a one-operation native batch. Text patches must match their captured original content without ambiguity unless native replace_all is explicit. Support-file changes, multiple-operation batches, new skills and unrelated proposals stay in Hermes' ordinary pending mechanism. The selected local oracle is trusted operator code, not a model tool or a model-selected command. It receives the playbook text and a phase and must bound its own task execution. It returns explicit case IDs and boolean outcomes.

Baseline and candidate must run the same cases; every candidate case must pass and at least one must improve. The adapter records task evidence, the proposal hash, oracle identity and before/candidate files in the native skill ledger, then calls native apply. It repeats the task after activation, permitting additional held-out checks while requiring all original cases. A failed or unavailable initial check invokes native rollback. --audit EVALUATION_ID repeats an existing candidate check later, using the same oracle implementation. Once the ledger records a completed activation, an unavailable later audit records unavailable and retains the last qualified files. A completed task regression still invokes native rollback. Infrastructure failure is not a failed task case. This is an operator/cron entry point, not an additional background service.

The proposal records its original file hash. A stale proposal or later owner edit is held rather than overwritten. A candidate ledger entry is written before apply and remains a recovery target if the process exits between apply and evaluation; retrying the pending ID resumes its post-check. Rollback checks all captured current files against the candidate snapshot first and restores only native skill files. It does not rewind memory, permissions or action records. These byte checks do not lock editors or foreign processes, so select a curator-owned path without another concurrent writer for automatic rollout. Native curation usage counters remain native telemetry after file rollback.

Qualification tests exercise real pending files, native curator ownership, mutation/rollback and ledger recovery with controlled task outcomes. A selected oracle only establishes performance on its declared task cases. No evaluator or automatic rollout is enabled by default, and this contract does not qualify arbitrary code changes or prove general self-improvement.

A disposable local-inference qualification also exercised an actual native review and exact text patch: its declared task scored 0/2 before, 2/2 with the candidate, and 2/2 after activation. An injected evaluator outage restored the original bytes through native rollback. This was a neutral fixture, not a live recurring skill or an observed spontaneous model regression.