galet-memory

Provider-neutral memory abstractions and reusable implementations for agent applications.

This package is being extracted from Lucy's src/coala_memory package. It will own memory models, interfaces, retrieval and ranking logic, and reusable persistence implementations.

Dependency rule

Applications may depend on galet-memory; galet-memory must never depend on an application.

In particular, this repository must not import from Lucy's src package. Lucy-specific configuration, dependency injection, storage adapters, agent lookup, request context, and tool handlers remain in Lucy.

Planned extraction order

1. Neutral episodic, semantic, procedural, and working-memory interfaces and models.
2. Retrieval, ranking, and digest logic.
3. Reusable SQLite and vector-backed implementations behind package-owned ports.
4. Lucy compatibility re-exports and adapters.
5. Exact-commit integration in Lucy followed by parity and full-suite testing.

The extraction must preserve database formats, retrieval results, prompts, and tool permissions.

Storage ports

Reusable memory implementations depend on narrow, structurally typed ports:

• EmbeddingProvider turns text into vectors. GaletEmbeddingProvider adapts Galet's EmbeddingApi.
• EmbeddingIndex performs namespace-scoped similarity queries without exposing an application's storage records.
• TextLoader loads bounded text from paths already authorized by the host.
• ContextRepository returns neutral context and skill snapshots.

Host applications adapt their storage and configuration to these ports. The package includes VectorSemanticMemory, EmbeddingDigestRecall, ContextProceduralMemory, and a basic FileTextLoader.

Request-scoped embedding reuse

Semantic document recall and episodic digest recall commonly embed the same query. Wrap their shared provider once and open a cache scope around the application request:

from galet_memory import CachingEmbeddingProvider

embeddings = CachingEmbeddingProvider(base_embeddings)
semantic_memory = VectorSemanticMemory(embeddings=embeddings, ...)
digest_recall = EmbeddingDigestRecall(embeddings=embeddings, ...)

with embeddings.request_scope() as cache:
    digest_recall(episodic_request)
    semantic_memory.recall(semantic_request)

print(cache.info())

The cache key contains the exact embedding model and the exact ordered input texts. A model change therefore cannot reuse vectors from the previous model. Calls outside a request scope pass through uncached, concurrent contexts are isolated, and failed provider calls are never stored.

SqliteVecEmbeddingIndex reads the existing 1536-dimension sqlite-vec schema. It defaults to vec_embeddings_v2 and joins results to embedding_metadata; the original vec_embeddings table can be selected explicitly for legacy reads. Internal tables generated by vec0 are never accessed directly.

SqliteEpisodicMemory implements both the prompt-time EpisodicMemory and session-management EpisodicMemoryManager contracts. It opens the existing Lucy-compatible kv/logs schema directly, while exposing only neutral galet-memory models. Existing keys under sessions/ and correlations/ are preserved, so adopting the package does not require a database migration.

JsonlEpisodicMemory implements the same contracts over a filesystem root. It preserves Lucy's existing sessions/<id>/meta.json, sessions/<id>/events.jsonl, and correlations/<id>.jsonl layout. Applications can therefore select SQLite or JSONL at their composition root without their handlers, endpoints, or prompt compiler knowing which medium is in use.

from galet_memory import SqliteEpisodicMemory

episodic = SqliteEpisodicMemory("/path/to/chat2.sqlite")
session = episodic.get_session("existing-session-id")

# Or use the existing Lucy-compatible JSONL directory.
from galet_memory import JsonlEpisodicMemory

episodic = JsonlEpisodicMemory("/path/to/storage/data/chat2")

Episodic memory road test

Create a disposable database and session:

galet-memory-episodic --db /tmp/galet-chat.sqlite create \
  --account demo --agent lucy --session-id road-test \
  --friendly-name "Road test"

Append and inspect an event:

galet-memory-episodic --db /tmp/galet-chat.sqlite add \
  road-test "Hello episodic memory"

galet-memory-episodic --db /tmp/galet-chat.sqlite show road-test

Append supplied digest text as a logical archive boundary, then compare the visible and complete histories:

galet-memory-episodic --db /tmp/galet-chat.sqlite archive \
  road-test "The earlier conversation was summarized." --account demo

galet-memory-episodic --db /tmp/galet-chat.sqlite show road-test
galet-memory-episodic --db /tmp/galet-chat.sqlite show road-test --scope all

The sample accepts digest text directly and does not call an LLM. Run it against a disposable database or a copy while experimenting.

Embedding memory road test

Install the package, set OPENAI_API_KEY (or use Galet's GALET_CREDENTIAL_PATH), and point the CLI at a copy or test embedding database.

Add a sample:

galet-memory-embeddings \
  --db /home/junwin/lucy_storage/data/embeddings-v2.sqlite \
  --account junwin \
  --namespace demo \
  add "The allotment has runner beans and three apple trees."

Query it:

galet-memory-embeddings \
  --db /home/junwin/lucy_storage/data/embeddings-v2.sqlite \
  --account junwin \
  --namespace demo \
  query "What fruit trees are in the allotment?"

Query it with path to credentials:

galet-memory-embeddings \
  --credential-path /home/zzzzzz/credential \
  --db /home/junwin/lucy_storage/data/embeddings-v2.sqlite \
  --account junwin \
  --namespace demo \
  query "What fruit trees are in the allotment?"

Use --extension when vec0 is not installed at /usr/local/lib/sqlite-vec/vec0.so. The current schema requires 1536-dimension vectors, so the default model is text-embedding-3-small. Run against a copy of a production database when experimenting.