eidetic-cli 0.12.0

Agent/CLI providing eidetic perfect-recall memory

eidetic-cli

Agent/CLI providing eidetic perfect-recall memory

What you get

• An agent-first CLI cited from teken (afi-cli) — the runtime declares data-refinery-cli[store] as its storage dependency; neo4j and pymongo arrive transitively via that extra. Consumers stay dependency-free because they call eidetic over a subprocess boundary.
• A mesh identity — culture.yaml (suffix + backend) and the matching prompt file (CLAUDE.md for backend: claude).
• The canonical guildmaster skill kit under .claude/skills/, vendored cite-don't-import, plus eidetic's own first-party remember / recall skills (a shared ~/.eidetic/memory store both Claude and the colleague backend can drive). See docs/skill-sources.md.
• A build + deploy baseline — pytest, lint, the agent-first rubric gate, and PyPI Trusted Publishing wired into GitHub Actions.

Quickstart

uv sync
uv run pytest -n auto               # run the test suite
uv run eidetic whoami              # identity from culture.yaml (console script is `eidetic`, not `eidetic-cli`)
uv run eidetic learn               # self-teaching prompt (add --json)
uv run teken cli doctor . --strict  # the agent-first rubric gate CI runs

CLI

Verb	What it does
whoami	Report this agent's nick, version, backend, and model from culture.yaml.
learn	Print a structured self-teaching prompt.
explain <path>	Markdown docs for any noun/verb path.
overview	Read-only snapshot of the agent plus a live Store section covering all stores: per-backend record counts + live/unavailable status (files/mongo/graph), per-scope + lifecycle breakdown, link-connections, and distinct contributors per scope (union of each record's added_by and metadata.author). Narrow with --backend/--scope. A down backend degrades to unavailable via a fast probe (never crashes).
doctor	Check the agent-identity invariants (prompt-file-present, backend-consistency).
remember	Ingest memory records — one JSON object or NDJSON on stdin; idempotent upsert by id; stamps created date; auto-stamps added_by (resolution: --added-by flag > culture.yaml mesh nick > None); accepts supersedes/links; --backend/--scope/--visibility.
recall <query>	Search the store — top-k hits with text + full metadata + score + signal; scope-aware (no private→public leak). Four --modes: exact (substring), approximate (vector), keyword (BM25), hybrid (blend, default; --alpha). Lifecycle flags: --include-shadowed, --include-archived (both excluded by default). Plus --top-k/--filter/--backend/--case-sensitive.
sweep	Apply lifecycle transitions (shadow/archive) across the whole store — never deletes, only flips lifecycle. Supports --dry-run.
migrate qq	One-shot idempotent import of legacy QQ memory (core.md/notes.md, MongoDB, Neo4j) into a private scope.
migrate store	One-shot idempotent upgrade of an on-disk store from legacy Record-JSONL to Envelope-JSONL. Delegates the rewrite to data-refinery's store.migrate endpoint — eidetic constructs no filesystem write path. --dry-run/--data-dir.
cli overview	Describe the CLI surface itself.

Every command supports --json. Results go to stdout, errors/diagnostics to stderr (never mixed). Exit codes: 0 success, 1 user error, 2 environment error, 3+ reserved.

Storage

The files backend (default) is self-contained — JSONL on disk, no services needed. For the mongo / neo4j backends, the storage stack is owned and published by data-refinery-cli as the GHCR image ghcr.io/agentculture/data-refinery-stack. Bring it up with:

data-refinery stack up              # mongo on host :27018, neo4j bolt :7687 / UI :7474
eidetic remember --backend mongo …  # eidetic's default connection settings already match

The --backend token is uniform across every verb: files, mongo, neo4j, or graph (graph is an alias for neo4j, kept as the operator-preferred display label) — so a driving agent can reuse one token everywhere.

eidetic depends on data-refinery-cli[store] (0.6.x) and imports data_refinery.store / data_refinery.quality for storage operations. It keeps all memory semantics — the record schema, recall ranking modes, scoring, freshness signal, and no-hard-delete lifecycle — while the storage substrate is owned by data-refinery-cli. Storage mechanics never cross the boundary: even the on-disk format upgrade (migrate store) is delegated to data-refinery's store.migrate endpoint, so eidetic constructs no filesystem write path of its own. (Migration tracked in eidetic#13 / data-refinery-cli#1; the migrate endpoint in #8.)

Embeddings + rerank come from a separate HTTP endpoint — the local lobes fleet gateway, which fronts every role (embedder, reranker, cortex, …) on one OpenAI-compatible port. The per-role vLLM containers are not published to the host, so a per-gear port is always wrong; lobes endpoint embedder reports the live value.

Variable	Default
EIDETIC_EMBED_URL	http://localhost:8001/v1
EIDETIC_EMBED_MODEL	Qwen/Qwen3-Embedding-0.6B
EIDETIC_RERANK_MODEL	Qwen/Qwen3-Reranker-0.6B
EIDETIC_EMBED_API_KEY	unset — falls back to COLLEAGUE_API_KEY, then CULTURE_VLLM_API_KEY

The gateway routes on the request's model field, which is why embed and rerank name different models. It also enforces a bearer token: with no key set the request 401s and eidetic degrades to a deterministic local lexical fallback — recall still answers, but not semantically. Only approximate/hybrid recall use the endpoint; exact/keyword are pure lexical and work fully offline.

Borrowed credentials are scoped. EIDETIC_EMBED_API_KEY is eidetic's own variable — set it and it is sent wherever you point EIDETIC_EMBED_URL. The other two belong to sibling tools and are borrowed so a configured box needs no extra setup; because you never paired them with eidetic's endpoint, they are sent only to a loopback or https:// URL and withheld from a cleartext remote host (with a one-time warning on stderr). lobes tunnel publishes over HTTPS, so remote deployments are unaffected. Passing api_key="" explicitly disables auth regardless of the environment.

Reference deployment. The default above is the reference rig's real embedder, and it is a single agreed value across every surface — the eidetic/memory/embed.py code default, the vendored recall.sh/remember.sh wrapper exports, and this README (eidetic-cli#28 / colleague#293, drift-tested by tests/test_embed_default_drift.py; see docs/contract.md for the machine-readable record). Agreement alone is not correctness: #28 aligned all three surfaces on :8002, a per-gear container port never published to the host, so every surface agreed on an endpoint that always refused — and the lexical fallback hid it. Check changes against the live fleet (lobes endpoint embedder), not just against each other. On the AgentCulture mesh, the same endpoint is discoverable rather than hardcoded: the lobes gateway serves an embedder role in its GET /capabilities response (lobes-cli >= 0.38, where each role's endpoint field is client-reachable — see colleague's colleague/lobes.py), and a consumer such as colleague resolves that role and injects EIDETIC_EMBED_URL/EIDETIC_EMBED_MODEL into the environment before shelling out to eidetic. That injection never overrides an operator's own explicit export: EmbedClient.__init__ (eidetic/memory/embed.py) reads os.environ.get("EIDETIC_EMBED_URL"/"EIDETIC_EMBED_MODEL") ahead of the built-in default and no CLI flag intervenes between the two, so an explicitly-set env var always wins — eidetic's own env-first resolution order already guarantees this; lobes discovery just supplies a smarter default for the same environment slot, never a competing one.

Freshness signal

Every record carries temporal state used to compute a freshness signal — a float in [0, 1] that blends into recall ranking so recently-created and frequently-recalled records surface ahead of stale ones:

• created — ISO-8601 date stamped at remember time; drives the age-decay factor (1/(1 + days_old * DECAY_RATE)).
• last_recall + recall_count — updated passively on every recall hit; drive an access bonus (capped at +0.5) and a staleness penalty (days_since_recall * DECAY_RATE).
• links — related-memory references; reserved for a future corroboration term (weight is currently 0.0, the hook is wired).

The signal is computed at recall time and exposed as signal in every hit alongside score. Records with no temporal data (undated legacy records) pass through unmodified — the blend is an exact identity for them.

The blend is multiplicative around the neutral midpoint (SIGNAL_BLEND_BETA = 0.25), so a fully neutral signal is a no-op and only records carrying real temporal data move in rank. The formula:

access_bonus = min(0.5, recall_count * 0.05)
age_factor   = 1 / (1 + days_old * 0.01)
staleness    = days_since_last_recall * 0.01
signal       = clamp((0.5 - staleness + access_bonus) * age_factor, 0, 1)
blended_score = score * (1 + 0.25 * (signal - 0.5))

Attribution (added_by)

Every record carries an added_by field that identifies the agent or caller that ingested it. eidetic remember auto-stamps the field when it is absent from the record JSON, using this resolution order:

1. --added-by <value> — explicit override on the CLI flag.
2. culture.yaml mesh nick — the suffix declared in the repo's culture.yaml; this is the normal case when running inside a mesh agent.
3. None — when no culture.yaml is present (e.g. a wheel install or a bare subprocess call without a repo context).

An explicit added_by value already present in the record JSON is always preserved verbatim — remember never overwrites a caller-supplied attribution. The field is None for legacy records that pre-date this feature.

eidetic overview (and overview --store --scope <name>) reports distinct contributors per scope: the union of each record's added_by and any metadata.author value, deduplicated and sorted.

# Ingest as a named caller:
eidetic remember --added-by my-agent '{"id":"r1","text":"hello","type":"note"}'

# Override the mesh nick for a bulk import:
echo '{"id":"r2","text":"world","type":"note"}' | eidetic remember --added-by importer

# View contributors per scope:
eidetic overview --scope default

Lifecycle (no hard-delete)

eidetic never deletes a record. Records move through a lifecycle state machine:

State	Meaning
active	Default; visible in recall results.
shadowed	Superseded within the same scope by a newer record that declares supersedes. Retrieved only with --include-shadowed.
archived	Older than ~1 year (created) or signal below threshold (0.25). Retrieved only with --include-archived.

Transitions are applied by eidetic sweep (the only command that writes lifecycle changes). --dry-run reports without writing. Records with metadata.protected set to a truthy value are exempt from all transitions.

Within-scope shadowing only. A supersedes link only shadows its target when both records share the same scope (name and visibility). Cross-scope links are ignored, preserving the public/private no-leak invariant.

Ingest with supersedes and links:

# New version of a record shadows the old one (same scope required):
eidetic remember '{"id":"r2","text":"...","type":"note","supersedes":"r1","links":["r3","r4"]}'

Then run eidetic sweep to apply the transition: r1 gets lifecycle=shadowed, r2 stays active.

Migrate QQ memory

eidetic migrate qq performs a one-shot idempotent import of the legacy QQ (Claude's personal) memory stack into a private eidetic scope:

• Sources read: ~/.claude/skills/memory/references/core.md and notes.md (one record per ## section), MongoDB claude_notes collection, Neo4j entities tagged knowledge_context="claude".
• Destination: --scope qq --visibility private by default — personal data never leaks into a public recall.
• Idempotent: stable per-source ids (qq-file:<path>#<section-slug>, qq-mongo:<id>, qq-neo4j:<id>) make re-runs safe.
• Resilient: a down Mongo or Neo4j is skipped with a warning, not fatal.

eidetic migrate qq --json           # migrate from all sources, JSON report
eidetic migrate qq --backend mongo  # store into eidetic's mongo backend
eidetic migrate qq --file ~/my.md   # restrict to a specific markdown file

Known limitations (tracked follow-ups): --filter is exact string-equality on metadata (time-range filtering is future work); the files backend re-embeds candidates per search (no embedding cache yet); the Neo4j backend fetches nodes and ranks in Python (vector-index pushdown is future work).

Make it your own

1. Rename the package eidetic/ and the eidetic-cli CLI/dist name throughout pyproject.toml, the package, tests/, sonar-project.properties, and this README.md. The name is hard-coded in ~30 files, so list every occurrence first — see the git grep discovery command in CLAUDE.md, the authoritative rename procedure.
2. Edit culture.yaml with your suffix and backend.
3. Rewrite CLAUDE.md for your agent and run /init.
4. Re-vendor only the skills you need from guildmaster (see docs/skill-sources.md).

See CLAUDE.md for the full conventions (version-bump-every-PR, the cicd PR lane, deploy setup).

License

MIT — see LICENSE.