SerenCorpusCallosum

The callosum. A read-only fan that federates N memory stores into one ranked recall surface. Left brain (SerenLoci - structured facts) plus right brain (SerenMemory - episodic memory) plus however many more you hook in - merged into a single ordered list you can hand to a model.

It owns no store of its own. It remembers nothing. It only fans, floors, and merges what the hemispheres hand back. That read-only-by-construction shape is the whole point: the callosum is glue, not a place data lives.

Why Reciprocal Rank Fusion (and not "just sort by score")

Every store in this family can change its embedder independently - both SerenMemory and SerenLoci ship embedder-migration. The moment two stores run different embedders, their raw distances and scores live in different, incomparable number spaces. Sorting a merged list by those scores is comparing apples to a slightly different apple every time someone migrates a model.

So the callosum never reads magnitudes. It reads only each store's rank ordering - position 1, 2, 3 within its own results - and merges with Reciprocal Rank Fusion:

score(hit) = weight_of_store / (k + rank_in_store)      # k = 60

That's embedder-agnostic by construction: rescale any store's scores however you like, the order is untouched, so the merge is untouched. There's a test that proves exactly this (test_embedder_change_does_not_perturb_order) - multiply one store's magnitudes by an arbitrary factor and the fused ranking comes out byte-identical.

Two per-store knobs decide how much a store is trusted (the federation has fifteen more; GET /stores lists the ones POST /configure honours as supported_knobs):

• weight (per store) - the one cross-store trust lever. Trust facts more than episodes? Give Loci a higher weight.
• floor (per store) - a relevance floor applied before fusion, so "rank 1 of a bag of garbage" can't sneak to the top. Default 0 (trust the store's own ordering); raise toward ~0.3 if a store is noisy.

Implementation architecture

Federation engine

The Federation class owns the fan-out. On a search request it:

1. Fans out - dispatches the query to every active store in parallel, each via its adapter, with a per-store timeout.
2. Floors - discards hits whose base_relevance is below the store's configured floor before ranking.
3. Ranks - sorts each store's survivors by native score descending, assigns rank order.
4. Fuses - applies RRF across all stores: Σ weight / (k + rank_in_store).
5. Applies authority margin - if authority_margin > 0, an exact-key hit (native_score = 1.0) gets boosted above the #1 RRF-ranked result by that margin.
6. Truncates - returns the top n_results.

A slow or down store degrades the result but never takes the call down with it - and the response says so: stores_searched is the stores that answered (an empty answer counts), failed names each store that was fanned and did not answer with why (timeout, refused, an HTTP error), and skipped is a store that was never bound (a config error). A timeout and "knows nothing" are different facts.

7. Satellite edges - a SerenMemory core hit carries metadata.surroundings (satellite count, what it superseded); its newest satellites and the superseded core are appended after the packet as marked edges (source: satellite-edge / supersedes-edge, edge_of = the core, score 0.0), budget satellite_budget shared round-robin across the packet's cores, before the topic edges. The over-fetch each store is asked for is clamped to what its contract accepts (Memory: 50).

Adapter layer

Each store type has a corresponding adapter class. Adapters translate the store's native response shape into a uniform StoreHit:

One adapter covers every store that speaks the same protocol - adding a SerenMemory-speaking store is a config entry, not a code change.

Overlay system

Stores come from two sources merged at startup:

• Base stores - declared in the hand-authored yaml config. Config-owned; the Bridge UI won't delete them (it points you back at the file instead).
• Managed stores - added via the Bridge viewer's + Add store form, persisted in a separate runtime-stores.json overlay file. Removable from the UI with the ✕ button.

Base always wins a name collision, so the overlay can never quietly shadow something you hand-wrote. The overlay path defaults to a sibling of the config file; override with SEREN_SCC_RUNTIME_STORES env var.

Edges system (optional)

When edges_enabled: true, the federation appends topic-association edges after fusion. For each hit in the fused list, the edges system looks up topically related entries from other stores and appends them as additional results (up to edge_budget). This surfaces cross-store connections - a Loci fact about a project's coding convention might pull in a Memory entry about a related discussion.

Edges are purely additive: they never replace or re-rank the fused list, only extend it. A disabled edges config or a store that doesn't support topic queries degrades gracefully.

Dynamic runtime configuration

Federation parameters can be tuned at runtime without a restart via POST /configure:

POST /configure
{
  "k": 30,
  "fusion_mode": "rrf_pct",
  "authority_margin": 0.1,
  "min_per_store": 3,
  "edges_enabled": false,
  "edge_budget": 0,
  "n_results": 25,
  "fetch_multiplier": 4,
  "per_store_timeout_s": 10.0,
  "stores": [
    {"name": "facts", "weight": 2.0, "floor": 0.1}
  ]
}

All fields are optional - only supplied fields are changed. The live Federation is rebuilt immediately so the next /search picks up the new values. Per-store overrides mutate weight/floor on matching stores; an unknown store name returns 404, an invalid fusion_mode returns 422.

Tests

pytest tests/

The family

License

AGPL-3.0-or-later.