SKWhisper — the subconscious memory layer 🐧

Your agent dreams while it sleeps. SKWhisper is a quiet background daemon that reads your finished conversations, distills each one into a memory, notices what keeps coming up, and leaves a short briefing note your agent reads at the start of every new session — so it walks in already knowing what matters.

SKWhisper is the subconscious of a SKWorld sovereign agent. While the agent is talking to you (the conscious layer), SKWhisper works in the background: it watches session transcripts, summarizes idle ones with a local LLM, files the result into sovereign memory, tracks recurring topics/people, and regenerates a single context file — whisper.md — that gets injected into the next session. All of it runs on your hardware, against your Postgres, with no SaaS in the loop.

Inspired by letta-ai/claude-subconscious, rebuilt sovereign on the SKCapstone stack.

The 60-second version

flowchart LR
    SESS["you talk to your agent<br/>(a session transcript is written)"] --> IDLE["the session goes quiet<br/>(idle 5 min)"]
    IDLE --> DIGEST["SKWhisper summarizes it<br/>(local LLM)"]
    DIGEST --> MEM["files it as a memory<br/>(skmemory + skmem-pg)"]
    DIGEST --> PAT["updates the patterns<br/>(hot topics, people, questions)"]
    MEM --> CURATE["every 30 min: find what's relevant<br/>(hybrid vector + keyword search)"]
    PAT --> CURATE
    CURATE --> WHISPER["writes whisper.md<br/>(the briefing note)"]
    WHISPER --> NEXT["your next session starts<br/>already knowing the context"]

You never run anything by hand. Once the per-agent service is enabled, every new session automatically gets the subconscious context.

Quickstart

pip install -e .                              # into the ~/.skenv venv
SKAGENT=lumina skwhisper install --start      # install + enable the per-agent systemd service
SKAGENT=lumina skwhisper status               # tracked sessions, digest counts, daemon health
SKAGENT=lumina skwhisper digest --backlog     # process the entire pending session backlog now
SKAGENT=lumina skwhisper curate               # regenerate whisper.md immediately
SKAGENT=lumina skwhisper patterns             # show hot topics / people / repeated questions

The active agent is resolved from $SKAGENT (primary), then $SKCAPSTONE_AGENT (legacy), defaulting to lumina. Each agent gets its own daemon instance via the skwhisper@.service systemd template — one unit file, one instance per agent:

systemctl --user enable --now skwhisper@lumina
journalctl --user -u skwhisper@lumina -f

What SKWhisper provides

Where it lives in SKStack v2

SKWhisper is a Core capability — it sits next to identity and memory, and is the write path that keeps sovereign memory fed. It consumes session transcripts, leans on the Compute tier for the local LLM, and persists everything into Data (skmem-pg: pgvector + BM25 + the AGE graph). It is the background twin of skmemory: where the memory system is queried consciously during the ritual, SKWhisper fills it subconsciously between sessions.

flowchart TD
    SESS["session transcripts<br/>(~/.skcapstone/agents/&lt;agent&gt;/sessions/*.jsonl)"] --> SKW
    subgraph CORE["Core (governance · identity · memory)"]
      SKW["**skwhisper**<br/>watch · digest · pattern-track · curate"]
      SKMEMORY["skmemory<br/>(3-tier JSON memory + ritual)"]
      CAPAUTH["capauth<br/>(identity)"]
      CLOUD9["cloud9<br/>(emotional continuity)"]
    end
    subgraph COMPUTE["Compute (.100 GPU host)"]
      EMBED["ollama :11434<br/>(embed: mxbai-embed-large)"]
      SUMM["qwen3.6 OpenAI :8082<br/>(summarize + topic extract)"]
    end
    subgraph DATA["Data"]
      SKMEMPG["skmem-pg (Postgres 17)<br/>pgvector = vectors · pg_search = BM25 · AGE = knowledge graph"]
    end
    SKW -->|"embed digest"| EMBED
    SKW -->|"summarize + topics"| SUMM
    SKW -->|"snapshot (short-term)"| SKMEMORY
    SKW -->|"upsert vectors + graph edges"| SKMEMPG
    SKW -->|"hybrid search (vec + BM25 RRF)"| SKMEMPG
    SKW -->|"writes whisper.md"| RITUAL["skmemory ritual<br/>(injects context at session start)"]
    RITUAL --> SKMEMORY

Platform primitives it relates to

How whisper.md gets injected

The skmemory ritual (run by the session-start hook) reads ~/.skcapstone/agents/<agent>/skwhisper/whisper.md and prepends it into the context window before each session. No manual step — once the daemon is running, every new session automatically opens with the subconscious context already loaded.

Configuration

Per-agent config at ~/.skcapstone/agents/<agent>/config/skwhisper.toml (search order: explicit -c path → that path → ~/.config/skwhisper/ → ~/.skcapstone/config/). See config/skwhisper.toml for the full annotated template. Key knobs:

[ollama]
ollama_url      = "http://192.168.0.100:11434"   # EMBED endpoint (Ollama, /api/embed)
embed_model     = "mxbai-embed-large"            # 1024-dim; query MUST use the same model
summarize_url   = "http://192.168.0.100:8082"    # SUMMARIZE endpoint (qwen3.6 OpenAI server)
summarize_api   = "openai"                        # "openai" (default) | "ollama" (legacy /api/generate)
summarize_model = "qwen3.6"

[backends]
vector_backend  = "pgvector"            # pgvector (default) | qdrant | chromadb
graph_backend   = "age"                 # age (default) | falkordb | none

[watcher]
poll_interval   = 60                    # scan every 60s
idle_threshold  = 300                   # 5 min idle = ready to digest
min_messages    = 5                     # skip tiny sessions
skip_cron       = true                  # don't digest automated sessions

[curator]
curate_interval = 1800                  # regenerate whisper.md every 30 min
top_k           = 10                    # memories surfaced per curation

Backend selection also honors env vars (SKMEMORY_VECTOR_BACKEND, SKMEMORY_GRAPH_BACKEND, SKMEMORY_PG_DSN), so a fleet can be steered without touching per-agent TOML.

Digest model routing (embed vs. summarize)

Every digest makes two model calls, and as of 2026-06-17 they hit different endpoints:

Three new [ollama] keys drive the summarize side:

• summarize_url — http://192.168.0.100:8082 (the qwen3.6 OpenAI server)
• summarize_api — "openai" (default) or "ollama" (legacy Ollama /api/generate)
• summarize_model — "qwen3.6"

Why split them? The .100 host has a single 5060 Ti (16GB CUDA) running qwen3.6-27B (~13GB) — it is the only model that should run on that GPU. A separate small digest model has nowhere good to live: CUDA is full, the Intel Arc iGPU corrupts generated output over Vulkan (GGML_VK_VISIBLE_DEVICES=0 → garbage, confirmed), and a CPU-resident model saturates the host. So digests reuse the already-loaded qwen3.6 — zero extra VRAM, correct CUDA output, ~1.5s per short call. (Earlier summarize_model values — llama3.2:3b, then qwen3.5:4b — were both wrong: the first was removed from the host and produced a flood of 404s; the second spilled to CPU and saturated the box.)

An optional CPU-only fallback Ollama instance (deploy/ollama-digest.service, port 11436) exists for sites that want an isolated digest backend, but it ships disabled — qwen3.6 reuse is preferred.

CLI reference

skwhisper daemon                    # run the loop in the foreground (-v for verbose)
skwhisper digest                    # one digest cycle (respects idle_threshold)
skwhisper digest --backlog          # process ALL pending sessions, ignoring timing
skwhisper digest --backlog --batch-size 5
skwhisper curate                    # regenerate whisper.md
skwhisper curate --stdout           # print it instead of writing
skwhisper status                    # session breakdown + daemon health + top topics
skwhisper patterns [--json]         # hot topics / repeated questions / entities
skwhisper install [--agent X] [--start]   # write + enable the systemd template

Runtime state (per agent)

Documentation

Dependencies

• Python ≥ 3.11 (tomllib, asyncio)
• httpx — async HTTP to Ollama and the vector backends
• skmemory — SKCapstone 3-tier memory system
• A reachable embed endpoint — Ollama (local or on 192.168.0.100:11434) serving mxbai-embed-large — and a summarize endpoint — the qwen3.6-27B OpenAI-compatible server on 192.168.0.100:8082 (see Digest model routing above)
• A vector backend: Postgres + pgvector (default), or Qdrant, or ChromaDB

Part of the SKWorld sovereign ecosystem · the subconscious behind every agent · 🐧 smilinTux