agentevo-sdk

Client SDK, agentevo CLI, and agent adapters (Claude Code, Codex) for Agentevo — CI/CD for agent intelligence.

This README describes reserved SDK 0.2.4, local 0.1.8, core 0.1.7 source behavior. Final artifact/public installation gates are still open; an older public installer can supply different commands. From the repository use its development virtualenv; after the matching release is published and verified, install the isolated tool:

uv tool install agentevo-sdk==0.2.4

Run the following in your own project with a completed Claude Code session. skills install teaches your coding agent the loop below; run it first:

agentevo skills install
agentevo init
agentevo doctor
agentevo capture --latest
agentevo compile --latest
agentevo verify --latest --local --workdir .
agentevo promote --latest --to Verified --local
agentevo recall "fix flaky test" --capabilities Bash,Edit,Read
agentevo report

init creates private project labels and a local vault with a retained key; doctor reads status without repairs or provider calls. Initialized capture, latest selection and ordinary recall bind to this project. Explicit same-tenant workspace sharing is available; local labels do not restrict the OS owner. Initialized recall defaults to local memory with no login/cloud fallback. Outside an initialized project the legacy cloud default remains; use --offline there, or explicit --cloud for authenticated managed recall. Retrieval is advisory.

capture --latest selects Claude transcripts for this checkout; Codex accepts an explicit file with --adapter codex. Global flags (--workspace, --vault, --base-url, --key-provider, --device-id) precede the subcommand. Keep the OS keychain key, or provide a persistent 32-byte hex AGENTEVO_VAULT_KEY through headless secret management; do not generate a new key for each launch.

Inspect verification JSON: integrity, reported historical outcomes, fresh replay, manual review and task correctness are separate. Interactive verify can exit zero with failed/inconclusive results. It runs supported declared tests in the explicit workdir; tests are not sandboxed. Use the reviewed nonempty pinned agentevo ci --manifest FILE --workdir . for a strict gate. CI writes no ordinary promotable PASS history. Promotion lists exact results and refuses unmet gates; Trusted needs independent fresh evidence and explicit approval where required.

Supported Claude setup uses agentevo host setup claude-code to preview changes, then the same command with its reviewed --apply-preview-sha. It requires Claude 2.1.295+, grants no host permissions and compiles Candidates only. An explicit expiring selected test profile enables a bounded literal test family. Native Grep/Glob require one existing owned nonprivate file; broad discovery refuses. See the local setup guide for host profiles, current context, fresh drift/review recovery and exact limits.

Review observed activity and estimated value

agentevo report --json

report --json prints one JSON object (schema: agentevo.report/v1) built from this workspace's vault audit log; plain agentevo report writes the same numbers to an HTML file. Every number is labelled:

• measured: counts Agentevo recorded. Sessions captured, secrets redacted before storage (redactions_stored, by detector), objects by state and type, time from the first capture to the first Verified skill, recalls served and the recall hit rate, sessions that reused a skill, guard decisions and repeat mistakes caught (a learned guard stopped the change it was learned from). A recall hit only means recall returned something; whether it helped shows in reuse.after_recall. definitions spells out each measured number.
• estimated: a stated formula applied to measured counts, never presented as measured. Tool calls saved is, over reuse sessions that succeeded, max(0, tool calls of the session the skill was learned from minus tool calls of the reuse session). Active minutes saved is the same difference over active time (gaps of at most 10 minutes between events). Guard tool calls avoided is, per catch, the tool calls the guard's source session spent from the guarded change to the failure it caused. Each formula is repeated under estimated.formula.

The additive historical_reuse section reports signed call/time differences, coverage, unbound cases, adverse outcomes and exact held versions. Older gross fields above remain for compatibility. Neither is a controlled counterfactual or observed net provider bill. Actual savings remain unknown without attributable baseline/enabled charges, correctness, overhead and fees; a recall hit alone is not a saved task.

compile records reuse and lists it under reused. A session reuses a skill when it repeats the skill's edits and test runs in order (at least 2 of them, covering at least 60%) after the skill reached Verified or a recall returned it, or when it names the skill. To name it, run agentevo recall "<task>" --use <object-id> before or during the session (up to 30 minutes before its first event), or put agentevo:use <object-id> in the prompt. A skill never counts as reused by the session it was learned from.

Recall, guard, capture and compile write these audit entries locally and send nothing anywhere. They hold object ids, versions, states, decisions, counts, ranks and relevance scores, plus a keyed digest of the recall query. They never hold the query, prompts, commands, paths or tool output.

Raw data stays local by default. The cloud is opt-in:

agentevo login
agentevo sync mode --set searchable --terms-version 2026-09-21 --yes
agentevo sync push
agentevo recall "fix flaky test" --capabilities Bash,Edit,Read --cloud

login asks for a key from the console (API keys) and checks it against the cloud before storing it in your OS keychain; set AGENTEVO_PAT instead to skip the prompt (on a machine without an OS keychain, such as CI, the variable is all the CLI needs and login only checks it). login reports "stored": true only when the key reads back from the keychain; a keyring that keeps nothing (the null backend) gets "stored": false and a note that AGENTEVO_PAT is the key's source. The CLI talks to the hosted API, https://api.agentevo.sh; point --base-url (or AGENTEVO_BASE_URL) at a self-hosted server instead. sync push queues locally, and says so, when the cloud is unreachable. Pushing from a second or fresh vault lists versions the cloud already holds under already_present (exit 0). In a local_only workspace (the default, or after an owner downgrades it) sync push sends nothing and exits 1 with FORBIDDEN. The message names the exact consent command to run, and details.consent_command holds it too. Reading the workspace's sync mode is a billable request, so sync push and sync pull read it at most once every 10 minutes and remember it in the vault (a local_only workspace is read on every sync). sync mode --set makes the next sync read it again. The cloud checks the mode on every push: when an owner changed it elsewhere, the cloud refuses the push and stores nothing, and sync push reads the mode again and pushes once more.

On a server configured for browser device approval, use agentevo login --device to associate the login with this vault's signing key. Open the displayed console link, sign in as an owner/admin, review the code, device key, workspace and requested grant, then approve or deny explicitly. The CLI proves possession of the pinned key and stores the bearer/refresh pair only in the OS keychain; this mode requires a working keychain. It refreshes the same grant near expiry and refuses to reuse it with another vault. Device revocation withdraws this associated grant and its descendant credentials. Default PAT login and older unbound device flows remain separate: revoke their API keys explicitly. AGENTEVO_PAT takes precedence over stored device login; unset it to use the device grant. Local logout removes credentials from this machine; containment in the console is the server-side revocation action.

The Account page's Download account data action exports all cloud-held workspace data plus relevant account, usage, security and billing records, including handoffs and job/event data beyond list-page limits. It remains available at the Free quota. A tenant-wide human owner/admin can also call Client.export_account() using a key with control:read and sync:read. Signing secrets, credential hashes, temporary authentication/replay caches and webhook credential URLs are excluded. Local-only vault data and records held only by the identity provider are separate. Store exported files privately; retained security/billing data and backup expiry follow the disclosed policy.

Objects pulled onto another machine arrive as Candidates, because trust never transfers between machines. sync pull says so. recall withholds them, but counts matching ones under withheld_pulled_candidates, lists them under pulled_candidates, and gives the commands to run from your checkout: agentevo verify <object-id> <version> --local --workdir . replays the object's test command there, then agentevo promote <object-id> <version> --to Verified --local. Without --workdir the verify is inconclusive and says why: the evidence stays on the machine that pushed it.

Searchable sync sends sanitized step templates and fingerprints only. File contents, diffs, command output, prompts and absolute paths stay on your machine. Opaque Relay sends the full steps, end-to-end encrypted under a workspace key the cloud never sees: create it once, on the first machine, with agentevo sync key --init, and carry it to each other machine with sync key --export FILE there and sync key --import FILE (machines that share AGENTEVO_VAULT_KEY already share it). Sync identifies your machine by a random device id that the vault creates once; your username and hostname are never sent. --tenant your-org is only a label while you work offline: sync push sends a workspace's objects to the tenant you are logged in to and lists the ones it relabeled. Cloud rows show cloud_state. The managed copy reaches Trusted only with independent evidence from a device. One run is never enough for Trusted. See agentevo.sh for more.

Recall that finds you

recall --auto builds the query from the checkout it runs in, for agent hooks: repo name, branch, stack markers (lockfile and toolchain filenames only, never contents) and the runtime join the text, and the stack and runtime also travel as a cloud environment descriptor. The output echoes what was collected under context, so a hook can log exactly what the cloud saw. Nothing leaves the machine beyond coarse tokens: no file contents, no absolute paths, no environment values.

agentevo recall --auto "fix flaky test" --capabilities Bash,Edit,Read
agentevo recall --auto --failed-command "pytest -q" --error-file /tmp/pytest.log

--failed-command fingerprints secret flag values before sending (the same templates searchable sync stores); --error and --error-file send a DLP-redacted tail of at most 20 lines, and the two are mutually exclusive. A hook recipe: run recall --auto with the failing command and the tail of the error log before the agent retries. The JSON output carries objects (ranked hits with matchers, quality and freshness), plan with --plan, and gap naming query terms nothing covers, with the capture command that closes it. Exit codes are the CLI's usual contract: 0 with hits or an empty objects plus a hint, 1 with a {"code", "message", "details"} JSON error (not logged in, cloud unreachable, quota exhausted, no searchable sync).

Rate what came back and recall gets honest:

agentevo feedback skill:6d7abee7-f4d0-59bb-bac1-590e481321a4:v1 --verdict helpful
agentevo report --json

feedback stores ids plus the verdict on this machine only, never the query (pass --query and only its keyed digest is kept). Every later recall shows the totals under each hit's quality, and report counts helpful/unhelpful per object next to precision. Feedback never leaves the device; fleet-wide quality is a documented future, not this command.

agentevo digest briefs recall quality over recent days: hit rate, served vs applied per ranker variant, open gaps, and low-precision objects worth re-verifying (--days 1-365, --json for machines). Ranker variants are labels only — recall --ranker b tags the run, and report compares served/hit/apply per variant, so a new ranker earns its traffic.

Agent hosts without transcripts use the push path: hooks emit records one lifecycle event straight into this machine's vault through the same DLP + canonicalize path as capture (pull path stays recall --auto):

agentevo hooks emit --session "$RUN_ID" --event prompt --text "fix the flaky test"
agentevo hooks emit --session "$RUN_ID" --event tool_call --tool Bash --call-id c1 --cwd "$PWD"

Semantic recall is bring-your-own-key: the cloud stores and matches vectors but never calls a model. Set AGENTEVO_EMBEDDING_API_KEY (and optionally AGENTEVO_EMBEDDING_BASE_URL, an OpenAI-compatible endpoint) and name one model:

export AGENTEVO_EMBEDDING_API_KEY=key-from-your-provider
agentevo sync push --embed-model text-embedding-3-small
agentevo recall "flaky test on main" --capabilities Bash,Edit,Read --embed-model text-embedding-3-small

sync push --embed-model attaches one vector per pushed version, over the same minimized summary and steps the lexical index sees; the vector joins the content hash, so switching models later re-conflicts already-pushed versions instead of silently forking the index. recall --embed-model (or AGENTEVO_EMBEDDING_MODEL) embeds the query; hits that match by vector list semantic in matchers. Without a model both paths stay lexical-only. The key never appears in errors, logs, or audit data.

Reviewed bounded workflows

Trusted sources plus authored tool bindings compile Candidate workflows. A reuse nudge is an activity signal, never automatic trust or execution authority. In an initialized project name exact versions:

agentevo workflow compile SKILL_ID:v1 --bindings @bindings.json --rollback rollback.json
agentevo workflow install WORKFLOW_ID:v1

Installation writes private no-replace JSON data; same bytes are idempotent and conflicting data refuses. workflow note-run records a caller outcome, not an observed execution. Explicit workflow preview/run support existing text-file replacements with authored original restores, selected inputs/outputs, an actual closed execution policy and the exact reviewed digest. Run rechecks current trust, holds, context and policy before each phase, reports observed effects and handles cooperative cancellation/authorized rollback. Arbitrary Bash/Edit traces need explicit trusted adapter integration rather than implicit conversion. The bounded contract and recipe records the narrow supported family and final installed acceptance gate.

Team notifications

Shared recall is only half of team memory; the other half is knowing when it changes. The cloud emits verification.completed, verification.failed, promotion.approved, promotion.rejected and revocation.published (plus quota, incident and job events) to webhook endpoints a tenant admin registers. Every POST carries the event envelope (event_id, type, occurred_at, tenant_id, workspace, resource, payload), an X-Agentevo-Event-Id deduplication key, and an X-Agentevo-Signature header (t=<unix seconds>,v1=<hex HMAC-SHA256> over <timestamp>. plus the raw body). Verify before trusting, inside a 5-minute replay window:

curl -s -X POST https://api.agentevo.sh/v1/webhooks -H "Authorization: Bearer $AGENTEVO_PAT" -H "Content-Type: application/json" -H "Idempotency-Key: $(uuidgen)" -d '{"url":"https://hooks.example.com/agentevo","events":["promotion.approved","revocation.published"]}'

Prefer the native path: notify add --kind slack registers a subscription the cloud shapes to {"text"} itself (same HMAC scheme), and feed tails the same trust transitions in the terminal:

agentevo notify add --url https://hooks.slack.com/services/xxx --events promotion.approved,revocation.published --kind slack
agentevo notify list
agentevo notify test --config wh_abc123
agentevo feed --days 7 --limit 20

notify test delivers one webhook.test event without billing. Tenant-wide roles only: workspace-scoped keys get 403 on the feed, never sibling data.

Slack incoming webhooks want {"text": ...}, not the Agentevo envelope, so a generic subscription points at a tiny adapter. The stdlib-only sketch below verifies the signature, deduplicates on event_id, and posts one line per event; run it anywhere that receives HTTPS (a 20-line Cloudflare Worker reads the same way):

import hashlib, hmac, json, time, urllib.request

SECRET = "whsec_from_the_subscribe_response"
SLACK_URL = "https://hooks.slack.com/services/xxx/yyy/zzz"
seen: set[str] = set()


def handle(headers: dict[str, str], raw: bytes) -> int:
    sig = headers.get("x-agentevo-signature", "")
    try:
        parts = dict(item.split("=", 1) for item in sig.split(","))
        ts, presented = int(parts["t"]), parts["v1"]
    except (ValueError, KeyError):
        return 403
    if abs(time.time() - ts) > 300:
        return 403
    digest = hmac.new(SECRET.encode(), f"{ts}.".encode() + raw, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(f"t={ts},v1={digest}", sig):
        return 403
    event = json.loads(raw)
    if event["event_id"] in seen:
        return 200
    seen.add(event["event_id"])
    line = f"[{event['type']}] {event['resource']} ({event['workspace']})"
    urllib.request.urlopen(
        urllib.request.Request(SLACK_URL, data=json.dumps({"text": line}).encode(), method="POST")
    )
    return 200

Failed deliveries redrive automatically (5 attempts, doubling from 60 s) and stay inspectable per endpoint until they settle.

MCP + API for builders

Stdio defaults to the initialized project-local tools and needs no cloud login. Inspect that selected schema:

agentevo mcp tools --source local

Use the reviewed host setup claude-code procedure above for supported Claude hooks and project-local MCP registration. It supplies a constant absolute Python callback/root; it previews owned changes and preserves unrelated settings. Registration alone does not grant native permissions, verify/promote lessons or establish enforcement in another host. Other clients need explicit command, cwd, project scope and retained host-key/vault environment setup.

Explicit mcp serve --source cloud selects the authenticated cloud family and its own tools/schema. Cloud recall requires login/network/searchable consent and can spend a quota unit. A local failure never silently selects cloud.

Prefer raw REST instead: protected cloud calls need a scoped Bearer credential and an Idempotency-Key on POST mutations. The checked-in packages/agentevo-cloud/openapi/openapi.v1.json is the published v1 contract (regenerated from the live app, drift-tested), with recall at POST /v1/recall, capture at POST /v1/events/batch, and quota at GET /v1/usage (not billed).

Session handoff + repo maps

Explicit handoff snapshots preserve selected plan, progress and skill references for a resume brief you can paste into another session. handoff write captures a snapshot into the local vault (plan steps, done flags, skill refs, notes); handoff read renders the resume brief to paste into the fresh session. Bundles are sanitized at write time, so export/import move them across machines by file.

agentevo handoff write --title "Finish flaky fix" --step reproduce --step fix --done 1 --skills skill:abc
agentevo handoff read
agentevo handoff export --file handoff.json
agentevo handoff import @handoff.json

Repo maps record how a repo works so "how do I run e2e here?" gets the verified answer. Set one per checkout; recall --auto attaches the current checkout's map to its context automatically.

agentevo repomap set --test-cmd "pytest -q" --test-cmd "npm run e2e" --service api --owner team-web
agentevo repomap show

Handoffs and maps stay on the machine in this release; cloud-synced handoffs (no file step) are planned next.

Leaderboard and recorded use

leaderboard ranks this vault's skills by recorded uses and guards by catches, with recall/guard/reuse totals from the local audit trail. It works offline. report includes the same board in its HTML.

The legacy savings JSON field contains formula-based estimates of historical tool-call and active-time differences, plus estimated guard tool calls avoided. These estimates are not measured provider savings or a controlled comparison. Actual net money savings remain unknown without independently attributable baseline/enabled charges, task correctness, overhead and fees. See the report section above for formulas and signed historical comparisons.

--cloud ranks the tenant's actors by audited lifecycle work instead (promotions, verifications, compiles, ingested events); per-recall attribution stays on the device by design, so the team board never counts recalls.

agentevo leaderboard
agentevo leaderboard --limit 5
agentevo leaderboard --cloud --days 7

Community guards + fleet

community pull fetches the vendor-curated guard feed (exact-command blocks like rm -rf /, non-billable safety content). Guards store as Candidates — a signature proves authenticity, never trust — and --apply records your human grant to Verified, enforced under the default policy. Vendor guards never push back to the cloud.

compat is the fleet-compat matrix for one environment: every Verified/Trusted/Compiled object classified EXACT / COMPATIBLE / UNKNOWN / INCOMPATIBLE, with the drifted layers named. Re-check drifted rows locally, then re-confirm cloud standing:

agentevo community pull --tenant your-org --project web
agentevo community pull --apply --tenant your-org --project web
agentevo compat --runtime python3.12 --dep pytest=8.4.0
agentevo verify skill:aging 1 --revalidate

Cost, latency, and routing advice

advise analyzes one captured session on the machine (nothing is uploaded): token waste (--lens cost: oversized prompts, giant tool outputs, error-driven retries), serial waits and chattiness (--lens latency: slow round-trips, chatty tools, loop smells), and per-step model suggestions from the session's own outcomes (--lens route: erroring shapes to upsize, clean heavy shapes to try downsizing). Counts are estimated (chars/4, no provider usage rows); there are deliberately no dollar figures.

agentevo advise --latest
agentevo advise sess-abc --lens cost --lens route

Step graph

graph connects shared steps across Verified+ skills and workflows (Candidates never enter): implicit 3-step chains no single object runs ("a procedure nobody saved") and shortcuts where another object jumps a long run directly. Step identity is the fingerprinted command template, precise per value. Offline.

agentevo graph

License

Apache License 2.0 — see the LICENSE file shipped with this package.