Lighthouse memory

Agent memory on FalkorDB for structured events.

pip install lighthouse-graph

The package installs as lighthouse-graph and is imported as lighthouse. It needs Python 3.12 or later and a running FalkorDB. The tests run against FalkorDB 6.0.2: docker run -p 6379:6379 falkordb/falkordb:6.0.2 gives one.

The library stores what is true about entities, keeps every earlier version, and lets agents coordinate through the same graph. It has no LLM on the write path and no code for any one product. The port desk, in the same repository, is one application built on it.

Status: first version. Built and tested: versioned entities, relations with validity windows, time-travel reads, late rows put into history at their time, episodes with evidence, lessons, playbooks, promotion of what repeats into lessons and playbooks, handoffs, recall by the graph, by words, and by meaning, graph algorithms with answers that are kept until the graph grows, conversation history per user with summaries and erasure, write receipts and failure records, task claims with order and leases, partial updates, algorithms at one time, rows drawn from the text of entries, what relied on a fact or an entry that turned out wrong, the difference between two times and a what-if on top of one, trust in an agent from its record, one graph per tenant, and limits on every input. The full list is in Capabilities.

Example

from falkordb import FalkorDB
from lighthouse import Kind, ensure_schema, remember, history, as_of, related
from lighthouse import open_episode, record, close_episode, episodes_about
from lighthouse import hand_off, accept, handoffs_for
from lighthouse import search, pointing_at
from lighthouse import learn, lessons_about, write_playbook, playbooks_for
from lighthouse import groups, connection
from lighthouse import say, messages, said_about

ROOM = Kind("Room", key="code")
TEAM = Kind("Team", key="name")
DEVICE = Kind(
    "Device",
    key="serial",
    times=("last_seen",),
    text=("note",),
    relations={"LOCATED_IN": ROOM, "USED_BY": TEAM},
)

graph = FalkorDB(host="127.0.0.1", port=26379).select_graph("world")
ensure_schema(graph, [DEVICE])  # once, before the first write

remember(
    graph,
    DEVICE,
    by="inventory",
    rows=[
        {
            "key": "d1",
            "t": "2026-01-01T00:00:00Z",
            "values": {
                "status": "ok",
                "note": "pump room sensor",
                "last_seen": "2026-01-01T00:00:00Z",
            },
            "relations": {"LOCATED_IN": "r1", "USED_BY": ["red", "blue"]},
        }
    ],
)
# [{"key": "d1", "status": "created"}]

remember(
    graph,
    DEVICE,
    by="inventory",
    rows=[
        {
            "key": "d1",
            "t": "2026-01-02T00:00:00Z",
            "values": {"status": "hot", "note": "pump room sensor"},
            "relations": {"LOCATED_IN": "r2", "USED_BY": ["blue"]},
        }
    ],
)
# [{"key": "d1", "status": "changed"}]

history(graph, DEVICE, "d1")  # both versions, oldest first
as_of(graph, DEVICE, "d1", "2026-01-01T12:00:00Z")["status"]  # "ok"
related(graph, DEVICE, "d1", "LOCATED_IN")  # ["r2"]
related(graph, DEVICE, "d1", "USED_BY", "2026-01-01T12:00:00Z")  # ["blue", "red"]

# Recall: by the words of the text, and by what points at a thing.
search(graph, DEVICE, "sensor in the pump room", match="any")[0]["key"]  # "d1"
pointing_at(graph, DEVICE, "LOCATED_IN", "r2")  # ["d1"]

# What agents saw, forecast, and did about one event, and how it ended.
open_episode(graph, "d1-overheat", by="watcher", about=[(DEVICE, "d1")])
record(
    graph,
    "d1-overheat",
    "observation",
    by="watcher",
    values={"status": "hot"},
    cites=[(DEVICE, "d1")],
)
# {"id": "d1-overheat/1", "seq": 1, "status": "recorded"}
record(
    graph,
    "d1-overheat",
    "forecast",
    by="forecaster",
    values={"fails_in_hours": 6},
    cites=["d1-overheat/1"],
)
close_episode(
    graph, "d1-overheat", by="reviewer", outcome={"failed": False}, cites=["d1-overheat/2"]
)

episodes_about(graph, DEVICE, "d1")[0]["outcome"]["failed"]  # False

# What was learned from an episode that ended, and what to do next time.
learn(
    graph,
    "hot-in-r2",
    by="reviewer",
    statement="A hot device in room r2 did not fail.",
    supported_by=["d1-overheat"],
    about=[(DEVICE, "d1")],
)
lessons_about(graph, DEVICE, "d1")[0]["weight"]  # 1
write_playbook(
    graph,
    "hot-device",
    by="reviewer",
    trigger={"status": "hot"},
    steps=[{"do": "check the fan"}, {"do": "look again in an hour"}],
    supported_by=["d1-overheat"],
    needs=1,  # 3 by default
)
playbooks_for(graph, {"status": "hot", "room": "r2"})[0]["steps"][0]  # {"do": "check the fan"}

# Work passed to another agent: the handoff points at the record, it does not copy it.
hand_off(
    graph, "h1", by="reviewer", to="planner", points_to=[("Episode", "d1-overheat"), (DEVICE, "d1")]
)
handoffs_for(graph, "planner")[0]["points_to"]  # [("Device", "d1"), ("Episode", "d1-overheat")]
accept(graph, "h1", by="planner")  # {"id": "h1", "status": "accepted"}

# What was said, for each user, in order. A message names what it is about.
say(graph, "ann", "main", "user", "why is d1 hot?", by="ann", mentions=[(DEVICE, "d1")])
say(graph, "ann", "main", "helper", "the fan stopped", by="desk", mentions=[(DEVICE, "d1")])
# {"id": "main", "seq": 2, "status": "said"}
[m["text"] for m in messages(graph, "ann", "main")]  # ["why is d1 hot?", "the fan stopped"]
said_about(graph, "ann", (DEVICE, "d1"))[0]["text"]  # "the fan stopped": the newest first

# The graph as a whole: what belongs together, and how two things are connected.
groups(graph, [DEVICE])[0]["size"]  # 5: d1 with every room and team it ever had
connection(graph, (DEVICE, "d1"), (ROOM, "r1"))
# [{"from": ("Device", "d1"), "relation": "LOCATED_IN", "to": ("Room", "r1"), "forward": True,
#   "current": False, "since": 2026-01-01 00:00, "until": 2026-01-02 00:00}]

Model

flowchart LR
  E[Entity<br/>current values] -->|HAS_STATE| S1[State<br/>valid_from, valid_to]
  S1 -->|NEXT| S2[State<br/>valid_from]
  E -->|HAS_STATE| S2
  E -->|"RELATION {valid_from, valid_to}"| X[Entity]
  E -->|"RELATION {valid_from}"| Y[Entity]
  EP[Episode] -->|ABOUT| E
  EP -->|HAS| N1[Entry<br/>seq 1]
  EP -->|HAS| N2[Entry<br/>seq 2]
  N1 -->|NEXT| N2
  N2 -->|"CITES {as_of}"| N1
  N1 -->|"CITES {as_of}"| E
  A[Agent] -->|WROTE| N1
  A -->|GAVE| H[Handoff]
  H -->|TO| B[Agent]
  H -->|"POINTS_TO {as_of}"| EP
  T[Task] -->|"CLAIMED_BY {at}"| B
  T -->|AFTER| T0[Task]
  ST[State] -->|FROM| N1
  SE[Session<br/>of one user] -->|HAS| M1[Message<br/>seq 1]
  SE -->|HAS| M2[Message<br/>seq 2]
  M1 -->|NEXT| M2
  M2 -->|MENTIONS| R[Reference<br/>label, key]
  F[WriteFailure]

• An entity is a node with the label and key of its Kind, and the label Entity that all entities share. It holds the current values, so a query or an index on the entity sees what is true now.
• A State is one version of the entity's values. It has valid_from, and valid_to once a later version replaced it. States of one entity form a chain through NEXT. seen_last is the last time a row repeated the State, when one did after it began.
• A relation is an edge from one entity to another with its own valid_from and valid_to. A relation that did not change keeps its edge when another value changes.
• by on every State and relation edge names the agent or source that wrote it. recorded_at is the time the library stored it. valid_from is the time the fact became true.
• An episode is the record of one event. Its entries are numbered in the order they were written: an observation, a forecast, an action, or any type the application names. The entry of type outcome closes the episode.
• An entry cites its evidence: earlier entries, and entities. A cited entity is pinned to the time of the entry, so as_of() gives the State the agent saw. An Agent node has a WROTE edge to each entry it wrote.
• A lesson is what was learned from episodes that ended. It points at the episodes for it and against it, and at the episodes where it was used. A playbook is what to do when a situation comes again: a trigger, ordered steps, and the episodes it was drawn from. A new version replaces the one before; no version is changed.
• A handoff passes work from one agent to another. It points at entries, episodes, and entities, and holds no copy of them. Only its receiver can accept it, once.
• A session holds what was said with one user, as messages in the order they were written, and one summary of the messages up to a number. A session can be forgotten: it is the one thing the library deletes. A message mentions things by name: a Reference holds the label and the key of an entity or a record, and no edge to it, because the thing may be in another graph.
• A WriteFailure is a write that did not happen: the reason, the kind, the key, and up to 2,000 characters of the refused content. The same refusal again is counted on the same node, with times and last_at, so a bad row that a feed sends on every read does not grow the graph.
• An entity of kind Thing whose text has no vector of the current model has the label ThingToEmbed too, from the write that changed its text: embed() reads only those.
• A Stamp is one node with a random id that ensure_schema() gives a graph once. A graph that is deleted and made again under its name has another stamp: it is another graph. It also counts the entries stored in the graph, and keeps the marks of the jobs that read entries off the write path: how far each has come.
• Times are FalkorDB localdatetime values in UTC, in whole seconds.

The write rule

A row is the full picture of one entity at time t. What the row does not name is no longer true: a missing value is removed, a missing relation is closed.

remember() returns one receipt per row, in order.

Consequences of the rule:

• The same rows give the same history, however often they are sent. A replay of old rows is unchanged, not an error.
• Rows that arrive in another order give the history of their times. A late row ends the State it falls in at its time, and holds until the next State starts. The entity node keeps the current values: a late row is never the current State.
• The library keeps the changes, not every row. For each State it keeps seen_last, the last time a row repeated it. A late row with a time before seen_last is stale: the State held again after the row, at a time the library no longer knows exactly, so it is refused and stored as a failure, never placed wrong. A source that sends only changes never repeats a State, and its late rows are always inserted.
• When a late row says what the next State says, that State starts earlier. Its old start becomes its seen_last when it has none: a row said it then.
• A late row replaces the NEXT link between the two States it falls between. This is the one write that deletes, and the same query gives the graph a new stamp.
• A row can name the entry it comes from: {"key", "t", "values", "relations", "entry": "look/3"}, the id that record() returned for an observation. A State that the row adds points at that entry with a FROM edge, and history() and as_of() give its id as "entry". The entry is evidence, not content: the same values from another entry are unchanged, and the State keeps the entry it came from. Entries are never deleted, so the entry is checked before the write, in one read for the batch.
• No model decides what a new fact replaces. A value replaces the same value of the same entity. A relation target that is not in the new row is closed.

Partial updates. update(graph, kind, key, t, by, values=None, relations=None, entry=None) changes what it names from time t on, and keeps the rest as the current State says:

update(graph, DEVICE, "d1", "2026-01-01T13:00:00Z", by="agent", values={"status": "hot"})
update(graph, DEVICE, "d1", "2026-01-01T14:00:00Z", by="agent", relations={"LOCATED_IN": "r2"})

• A value of None is removed. A relation it names gets the targets it gives and keeps no other; a relation it does not name keeps its edges.
• It reads the current State, makes the full row, and writes it by the write rule, so its receipt is one of remember(). The row carries the State it was made from: if another writer added a State in between, nothing is written and update() reads again. A change is never written over one it did not see. After 5 reads that each found another writer, it is busy: refused and stored.
• It changes what is current. A time before the current State is stale, refused and stored: for an earlier time, send the full row to remember(). Two updates of one entity in the same second are a conflict, as two rows of one time are.

Tasks. A task is free when it is open, or when its claim ran out, and when every task it comes after is done. claim() gives the oldest free task of a kind in one query.

• A task can only come after tasks that exist, so the order of tasks has no cycle. A task waits while one it comes after is open or claimed, and is free once all of them are done.
• A claim with a lease holds that many seconds, at most seven days. After that, another agent can claim the task, and the first one can no longer complete it. While nobody has, the first one still can. There is no renewal: an agent that needs longer takes a longer lease.
• release() gives a task back. Every claim stays as a CLAIMED_BY edge; the task's owner names the last one.

Episodes

open_episode(), record(), and close_episode() each return one receipt.

• An entry is appended with one query. Entries written at the same moment get one order, and no number is used twice.
• Evidence must exist. An entry cannot cite an entry or an entity that is not in the graph.
• A refused entry uses no number. Its content is kept in the WriteFailure.
• An episode cannot be opened again after it is closed, and what an entry says cannot be changed. An entry that turns out wrong is marked with invalidate_entry(), not changed or deleted: see What relied on it.

Recall

Four ways to find what is in memory. The library holds no model: search by meaning uses a function that the application passes in.

fuse({"graph": ..., "words": ..., "meaning": ...}) merges the results into one ranking by reciprocal rank fusion. It uses the places only, because the scores of the three ways cannot be compared. Each result says which ranking put it where.

• similar() says why: each result has through, the (relation, target) pairs it shares. It compares the current relations of the entity with every relation the others ever had, so entities of the past are found.
• search() sees the current text of an entity. A value that changed is found by its new words only. search_history() finds the old words in the States, and says when each State held.
• The index never reads a text as it is. A node that can be searched keeps words: the letters and digits of its text, one space between them, and the index reads that. FalkorDB's index does not split words at a line break, a no-break space, or an underscore, so "first line\nsecond" indexed as it is would never be found by "line". history(), search(), episode(), and messages() do not show words.
• States of every kind share one index. search_history() keeps the States of an entity of the kind it was asked about.
• A question is never sent to the index as it is. The index has a query language (*, -, |, @field:, "..."); only the letters and digits of the question are sent, as at most 20 words. So * does not list every entity and @note:x does not choose a field.
• similar(), search(), and nearest() stop after 5 seconds. FalkorDB can stop a read.

Search by meaning:

MANUAL = Kind("Manual", key="code", text=("title", "body"), dimensions=384)


def embedder(texts):  # the application's model: a list of texts in, one vector for each out
    ...


embed(graph, MANUAL, embedder, model="name-of-the-model")  # off the write path, until waiting is 0
nearest(graph, MANUAL, "the seal of my pump leaks", embedder, model="name-of-the-model")

• embed() gives a vector to each entity whose text has none, at most limit in one call, and returns {"embedded", "waiting"}. It is the only place where the model is asked about stored text. remember() never asks it.
• embed() reads only the entities that wait: the write that gives an entity text, or changes it, gives it the label ThingToEmbed in the same query, and the write of its vector takes the label off. A label, because a write replaces every property of the entity and keeps its labels. A call costs what waits, not what the kind holds. After a change of model, the first call marks every entity with text once, a page at a time.
• A vector stands for the text it was made from. The write that changes the text of an entity takes its vector out of the index in the same query, so nearest() never answers from text that is gone. The entity is found again after the next embed(). A change of a value that is not text keeps the vector.
• A vector is stored only if the text is still what the model was asked about. A text that changed while the model worked gets no vector from that call.
• The name of the model is stored with each vector. Vectors of another model are never compared; embed() with a new name replaces them.
• What the embedder returns is input like any other: one vector for each text, of the declared size, of finite numbers, not all zeros. Anything else raises an error and nothing is stored.
• Entries get vectors in a graph whose schema gives them an index: ensure_schema(graph, kinds, entry_dimensions=384). embed_entries(graph, embedder, model) gives a vector to each entry with text that has none of this model; the text of an entry is its text values, in the order of their names, one per line. An entry never changes, so its vector stands for it for good. embed_entries() reads the entries stored since its mark for that model, by their serial; see Free text.

Algorithms

Four questions about the graph as a whole. FalkorDB computes the answers; the library names what it finds and bounds the cost.

groups(graph, [DEVICE], ["ABOUT"], records=["Episode"])  # the episodes about the same devices
important(graph, [DEVICE], "LOCATED_IN", limit=3)  # the rooms that most devices were in
between(graph, [PERSON])  # who stands between persons that do not know each other
connection(graph, (DEVICE, "d1"), (DEVICE, "d7"))  # through which rooms and teams

• Without at, they see every relation that was ever true. A relation edge is never deleted, and these searches cannot tell an open edge from a closed one. connection() says for each step whether the relation holds now, since when, and until when. Where several relations join the two things of a step, it tells of one that holds now, and otherwise of the one that ended last.
• groups() joins whichever way the relations point. With records, the records of the library take part: with records=["Episode"] and relations=["ABOUT"], a set is the episodes about the same entities, and those entities.
• important() ranks over the entities of every kind and the edges of one relation, or of every relation. The scores of all entities add up to 1. FalkorDB stops at a fixed precision: in the tests, a score is within 0.0005 of the exact value.
• between() follows the direction of the relations. An entity scores only when relations lead into it and out of it: a relation from a kind to itself, or a chain of kinds. Where one kind only points at another, every score is 0.
• between() counts paths from a sample of the entities and scales the counts up. Without sample, it follows 10,000,000 relations at most, from 32 entities at least: a graph that small gets the exact count, a larger one an estimate. On a test graph with hubs (6,000 entities, 30,000 relations), a sample of 32 found 7 of the true top 10, 128 found 8, and 256 found 9. Where the scores of all entities are close, no sample can order them.
• connection() follows relations in either direction. It follows the relations of the kinds of its two ends; kinds adds a kind that points at both, and relations=[..., "ABOUT"] lets it go through records.
• The first result of each is the oldest of equals: for the same graph, the same question gets the same answer.

At one time. With at, each of the four sees the relations that held at that time and the entities that existed then, and nothing else:

noon = "2026-01-01T12:00:00Z"
groups(graph, [DEVICE], ["LOCATED_IN"], at=noon)  # which devices shared a room then
connection(graph, (DEVICE, "d1"), (DEVICE, "d7"), at=noon)  # how they were joined then

• FalkorDB's algorithms cannot keep to the edges of one time, so the relations that held then are read in one query, and the algorithms run in the library: union-find, PageRank by the same rules as FalkorDB's, Brandes' count of paths, and a search for the shortest chain. For a time when every relation held, they give the answers of FalkorDB; the tests check that, and check each against a plain implementation on the relations the test wrote for that time.
• An entity exists from its first State. An entity that only relations point at exists while one of them holds.
• Records have no time: with at, no record takes part, and the edges of records are not followed.
• connection() searches from both ends, a step at a time from the end with fewer to look at, and reads only the neighbours of those. A step tells of a relation that held then; current and until say whether it holds now.
• between() here counts paths in Python: from 32 entities at least, and from as many as 2,000,000 steps allow. The seed picks them in another way than FalkorDB, so a sampled count differs from the one without at.
• One answer reads at most 1,000,000 relations or entities; more is refused. Answers at a time are not kept: a row that came late can change what held at a time without a node or an edge more.

One laptop, three runs each, with a script that is not in the repository. Reading the relations is 130 to 250 ms of each of the first three. Before the search for a chain read only the neighbours, connection() at a time took 600 ms; before the nodes were found by their id first, 970 ms.

An answer is kept until the graph grows. groups(), important(), and between() keep their answers in the process. The same question is answered from there while the graph has as many nodes and edges, of the labels and types that the question reads, as it had. A graph only grows, so equal numbers mean the same nodes and edges, and FalkorDB has those numbers at hand without a scan. The queries that delete, forget_session() and a late row that replaces the NEXT link between two States, give the graph a new stamp, and no answer from before is given again. A write that only changes values, or that adds episodes the question does not read, keeps the answer. important() without a relation reads every edge, so any new edge makes its answer new.

One laptop, three runs, with a script that is not in the repository. Before the best rows were picked ahead of naming them, and before the counts were queries of their own, the first answers took 90, 175, and 670 ms; between() then counted from 100 entities.

• Answers are kept only for a graph that has its stamp from ensure_schema(). The stamp tells a graph from another one made later under its name.
• Up to 64 answers are kept for one database client, each with up to 10,000 names. A caller gets lists of its own: to change them changes nothing that is kept.
• Delete nothing from a graph by hand. A kept answer does not see it.
• FalkorDB cannot stop these three searches while they run, and a write to the graph waits for them: a write sent during a search of 2 seconds waited 1.7 seconds. That is why each has a bounded cost, and why an answer is computed once.

Lessons and playbooks

The three kinds of memory are graph structures. An episode is what happened. A lesson is what was learned from episodes. A playbook is what to do. The library writes no lesson and no playbook of its own: an agent or a rule of the application says what was learned or what the steps are. The library keeps the evidence, counts it, and ranks by it.

Lessons. learn() stores a statement with the episodes it was learned from. Every one of them must have ended, and there must be at least one: a claim with no evidence is refused.

• The weight of a lesson is support - against + helped - failed: the episodes that agree, minus those that do not, plus the uses that led to a correct result, minus those that did not. A lesson that is contradicted or that fails sinks below at_least and is no longer given out.
• A lesson fades when nothing touches it: since leaves out lessons that no episode has supported, contradicted, or used since that time.
• Every count is changed in the same query that makes its edge. Eight agents that weigh one lesson at once lose no count, and an episode that eight agents add is counted once.

Playbooks. Writing what an agent will do is riskier than writing what it knows. So a playbook has four guards.

An entry can cite the lesson it used, and a handoff can point at a playbook: ("Lesson", id) and ("Playbook", "name#version") are references.

Promotion. What repeats in episodes that ended becomes a lesson or a playbook. The library cannot tell what a repeat is, so the application passes a rule: a function of one episode that says what the episode shows.

def rule(episode):  # what one episode that ended shows
    outcome = episode["entries"][-1]
    return [
        {
            "lesson": "d1-fan",
            "statement": "d1 gets hot because of its fan",
            "about": [(DEVICE, "d1")],
            "agrees": outcome["cause"] == "fan",
        },
        {
            "playbook": "cool-d1",
            "trigger": {"status": "hot", "device": "d1"},
            "steps": [{"do": outcome["step"]}],  # what fixed it
        },
        {"ran": "cool-d1", "succeeded": outcome["fixed"]},  # a playbook that ran in it
    ]


promote(graph, "reviewer", rule, at_least=3)
# {"episodes": 12, "lessons": [{"id": "d1-fan", "status": "learned", "new": 0}],
#  "playbooks": [{"id": "cool-d1#1", "version": 1, "status": "written"}], "runs": [...]}

• A lesson is learned when at_least episodes agree with it, from those episodes. After that, each episode for it or against it weighs it.
• A playbook is written when at_least episodes show the same trigger and steps. When more episodes show other steps than the newest version has, the next version is written. A playbook that was switched off is not written again with the same steps.
• A run is judged once, with ran(). Three failures in a row switch the playbook off.
• Every one of these counts an episode once, so promote() can run after every poll and writes nothing twice. A rule that gives something that is not a claim stops the promotion before it writes.
• The receipt of each lesson and playbook says what happened: learned, weighed, written, waiting (with how many episodes agree so far), unchanged, or the refusal of the library with the id it was for.

Handoffs

hand_off() and accept() each return one receipt.

• A handoff must point at something that exists. A handoff that only carries text is refused.
• A handoff given again with the same id changes nothing. Its receiver and what it points at cannot be replaced.
• accept() reads the receiver and marks the handoff in one query, so it is accepted once.

Conversations

What was said, for each user, in order. The functions take the graph that holds the conversations: with one graph for each tenant, that is the graph of the tenant.

• A session belongs to one user, and every call names the user. The key of a session holds the user and the id. So two users can both have a session main, and no id that one user asks for can reach a session of another. A session that the user does not have reads as empty.
• A message is added with one query: it takes its number and its place after the last message together. Messages written at the same moment get one order, and no number is used twice. A refused message uses no number.
• messages() finds each message by its key, so the last 20 of a session cost the same whether it holds 20 messages or 20,000: 0.1 ms in the database.
• A mention is a name, not an edge. mentions=[(VESSEL, 9000001)] stores the label and the key. The thing is not looked up: with one graph for each tenant and one that all share, it is in the other graph, and a query cannot span two graphs. Read the thing from its own graph by that key.
• said() sends only the letters and digits of a question to the index, as search() does. It sees the text of messages, not their values.
• The words of a message hold one more word for its user, from a hash of the name that is the same in every process, and said() asks the index for it. So the index gives the messages of the user who asks, not those of every user. The check of the user stays, since a message's text could hold another user's word.
• role is a name the application chooses, such as user and the name of an agent. by is who wrote the message. Both come from the caller.
• The time of a message, at, is when it was said. The order of the sessions of a user follows the time the library stored the last message, which a caller cannot set.

A long session is read as a summary and the messages after it:

found = context(graph, "ann", "main", last=20)
# {"summary": None, "upto": 0, "messages": [the last 20], "waiting": 37}
text = model_sums_up(messages(graph, "ann", "main", last=37, before=found["messages"][0]["seq"]))
summarize(graph, "ann", "main", text, upto=37, by="desk")
# {"id": "main", "upto": 37, "status": "summed"}
context(graph, "ann", "main")  # the summary, the 20 messages after message 37, and "waiting": 0

• The summary is the application's text: the library holds no model. context() says with waiting how many messages are neither summed nor given, so the application knows when to sum up again.
• A summary is not a message. messages() and said() do not give it.

Forgetting is the one place where the library deletes what was written. (A late row replaces the NEXT link between two States; see the write rule.) forget_session() deletes the messages of a session, the session with its summary, each reference that no other message mentions, and what was kept of its refused messages and summaries. forget_user() does that for every session of the user. After it, no property of any node or edge of the graph holds what the session held: a test reads them all.

• A session is deleted in steps of 200 messages, the last ones first, and the session counts down with them. So after every step it is a shorter session that reads right, and it goes with its last message. A message that is said while this runs goes too; one that is said after opens the session again at number 1.
• What a refused message held is kept in a WriteFailure, as for every refused write. Those of a conversation name their owner, so they are found and deleted with the rest.
• Other parts of the library rely on a graph that only grows: an answer of an algorithm is kept while the numbers of nodes and edges stay. Each query that deletes gives the graph a new stamp in the same step, so no answer from before is given again.
• What it does not reach: the name of a user where the application gave it as by of another write, and copies of the database outside it.

Free text

What agents write in entries can become rows. The library holds no model: the application passes the extractor, a function that reads one entry and gives the rows it finds.

def extractor(entry):  # the application's model, or plain code: one entry in, rows out
    return [{"key": "d1", "values": {"status": "hot"}, "relations": {"LOCATED_IN": "r2"}}]


extract(graph, DEVICE, extractor, by="reader", types=["observation"])  # off the write path
# {"read": 12, "rows": {"created": 3, "changed": 5, "unchanged": 4}, "waiting": 0}

• extract() reads entries with text that it has not read for the kind yet, oldest first, at most limit in one call. Each entry gets a READ_BY {kind, at, rows} edge to the agent that read it, so it is read once for each kind.
• Every entry takes a serial, the next number of the graph, in the query that stores it. FalkorDB runs one write of a graph at a time, so serials follow the order in which entries were stored, and a job that has seen serial n has seen every entry before it. A call of extract() reads the serials past its mark, through their index, a slice at a time, and moves the mark past what it did. The mark is kept on the graph's stamp, one for each kind and set of types. So a call costs what came since, also on a first run over a graph full of entries, and no entry is passed over.
• The rows are written by the write rule. Each State they add points at the entry with FROM, whatever entry the extractor named. A row without t is true from the time of the entry.
• What the extractor gives is input like any other: a row that cannot be stored is refused and kept as a failure, and the entry counts as read. An extractor that raises stops the call, and its entry waits for the next one. An answer that is not a list of at most 1,000 rows raises an error.
• Messages of conversations are not read: they can be erased, and facts drawn from them would stay. An application that wants them read writes what it takes from them as entries, where it can decide what may stay.
• With embed_entries() and nearest_entries(), the text of entries is found by meaning too.

What relied on it

A fact can turn out wrong: a sensor reported nonsense, or a row came late and rewrote what held. The graph knows what rested on it, because every entry cites its evidence, every State points at the entry it came from, and every lesson and playbook names its episodes.

• invalidate_entry() returns {"id", "status"} with invalidated (and impact), exists when it was marked before, or a refusal kept as a WriteFailure: unknown, invalid, or failed.
• A lesson is at risk when the episodes left for it would no longer carry it; a playbook, when fewer than three would.
• Only these functions and agent_trust() read the mark. Recall, lessons, and playbooks still use a marked entry: what to do about it is the application's to decide.

Two times, and what if

what_if() uses the union-find and the PageRank of the algorithms at one time, which the tests compare with plain implementations and with FalkorDB's.

Trust in an agent

agent_trust(graph, agent) counts what the graph records of an agent's work, for it and against it: the entries it wrote against those marked wrong, the closed episodes it wrote in that a lesson rests on against those that contradict one, the playbook runs it recorded that succeeded against those that failed, and the versions it wrote that held at least a minute against those replaced sooner. The score is the mean of the four rates, each smoothed as (for + 1) / (for + against + 2), over the kinds of evidence the agent has, so the many versions of a busy writer do not drown the rest. With no evidence, the score is None.

References

Episodes and handoffs point at things with references. episode() and handoffs_for() give references back in the same form, so what is read can be cited.

API

How the guarantees hold

Each one is a FalkorDB property the library relies on, checked by a test.

The suite runs against a real FalkorDB, on a throwaway graph per test. A mutation check with 1,088 deliberate bugs in copies of the library (846) and the port desk (242) was run; every bug made the suite fail, one of them by making a search run until the check stopped it. Two bugs lived in an earlier run of this check: a test that wrote the same refusal again and again, which the library now counts on one node, and a check in the PortWatch reader that grouping rows by day had made visible only in its count of skipped rows; each got a test. The check is in ../mutation/mutate.py: python mutation/mutate.py mutation/.work from the repository root, with nothing else writing to the database. A test that a copy cannot run, such as the one that compares this package with the documentation site, skips there.

FalkorDB behaviour the library works around

Found by test on the pinned image (graph module 60002, falkordb-py 1.7.1).

Capabilities

The target is the full capability set of a temporal knowledge-graph memory, with structured events as first-class input.

Limits and trust

The library is attacked in tests/test_attacks.py. The rule: hostile input costs the attacker one refused write, and costs nobody else anything.

What the library does not do:

• It does not know who is calling. by, the agent of a task, and the receiver of a handoff are names the caller gives. The application sets them from an authenticated identity, in server code. A name that comes from a model's output or a user's message can be any name.
• It does not decide who may read. A caller that holds a graph reads all of it. Give a tenant only its own graph.
• It does not limit how often a caller writes. Each call is bounded; the number of calls is not.
• It does not limit how often a caller asks a heavy question. groups(), important(), and between() cannot be stopped while they run, and writes wait for them. A question that was answered costs nothing until the graph grows; a new one costs a search. Do not let a model or a user choose their arguments without a limit on the calls.
• It erases conversations only. forget_session() and forget_user() delete what was said. Entities, episodes, lessons, and every other record are never deleted. Do not put there what a person may ask to have erased.
• It does not decide who may erase. forget_user(graph, "ann") deletes for any caller that holds the graph. Take the user from an authenticated identity, as for every other call.
• It does not judge what a message says. A message is text that someone wrote. Give it to a model as what was said, never as an instruction.
• It does not clean what it stores for display. Text is stored as given. Escape it where it is shown.
• It does not hide refused input. Up to 2,000 characters of a refused write are kept in the graph. Do not send secrets or personal data and rely on a refusal to drop them.

Limits of this version

• A row is a full snapshot. update() makes one from the current State; it cannot change the past, and two updates of one entity in the same second conflict.
• A late row whose time falls before a later repeat of the State it falls in is refused as stale. The library would need every row to place it.
• The late rows of a batch go in with a second query. 500 late rows took 79 to 90 ms, as long as 500 changes (77 to 92 ms); one late row into an entity with 1,000 States took 2.7 to 3.9 ms, against 1 ms for a change. One laptop. Before the late rows of a batch shared a query, 500 took 640 to 660 ms.
• One time resolution: seconds.
• A State keeps the values; relation history is in the edges. as_of() returns values, related() returns relation targets.
• When ensure_schema() adds a kind to a graph that is in use, stop the writers first.
• A batch is atomic per query of 500 rows, not across kinds.
• A State points at one entry, the one of the row that added it. A row that repeats it later from another entry adds no second link. Relation edges point at no entry: they start with the State of the same row.
• After a change of model, the first embed() marks every entity of the kind with text, a page at a time.
• playbooks_for() compares the triggers in the query, but reads every playbook that is switched on to do it. A value fits only a value of its own type: true does not fit 1.
• Who switched a playbook, and when, is kept for the last switch only.
• An agent that can write episodes can also close them and cite them. The number of episodes a playbook needs is a guard against haste, not against an agent that lies.
• nearest() sees the current text of entities. Past States are found by words only, with search_history().
• extract() asks the extractor about one entry at a time.
• A claim's lease cannot be renewed: an agent that needs longer takes a longer lease.
• A graph written by a version of the library without words has no words on its nodes: write the rows again to make them searchable.
• A session has one summary. The one before it is not kept.
• A session cannot be closed. It can be forgotten.
• forget_user() deletes in this graph. A user with sessions in several graphs is forgotten in each.
• A mention is not checked against any graph. A mention of something that does not exist is stored.
• ensure_schema() brings a graph that an earlier version wrote up to date, once: entries get serials in the order they were recorded, and messages get the word for their user. Stop the writers while it runs, as for any change of the schema.
• diff_between() and agent_trust() find what they read through indexes on when a State began and who wrote it; agent_trust() reads every playbook run to find the agent's.
• invalidate_entry() marks an entry; recall, lessons, and playbooks do not leave marked entries out.
• Without at, the algorithms see every relation that was ever true; connection() says per step what holds now. With at, they run in Python on what one query read: slower, and not kept.
• between() counts paths along the direction of the relations only.
• important() takes one relation or all of them, not a list.
• Answers are kept in the process that asked. Each process computes its own.
• A kept answer does not see what was deleted from the graph by hand, or a graph that was restored from an older copy while the process ran.

Layout

lighthouse-memory/
  lighthouse/
    __init__.py   the public names
    kinds.py      Kind, SELF, and the names the library owns
    clock.py      stamp(), now(): the one time format
    graphs.py     world(), tenant(), ensure_schema()
    write/
      rows.py     a row checked and made ready for a query
      batches.py  remember(): rows in waves and queries, each after the current State
      late.py     rows that came late, put into the history at their time
      partial.py  update(): some values or relations change, the rest stays
      failures.py record_failure(): what was not written is kept
    read.py       history(), as_of(), related()
    episodes.py   open_episode(), record(), close_episode(), episode(), episodes_about()
    extraction.py extract(): rows from the text of entries, by the application's extractor
    lessons.py    learn(), weigh(), used(), retire(), lesson(), lessons_about()
    playbooks.py  write_playbook(), playbooks_for(), ran(), switch(), playbook()
    promotion.py  promote(): what repeats in ended episodes, as lessons and playbooks
    handoffs.py   hand_off(), accept(), handoffs_for()
    algorithms.py groups(), important(), between(), connection()
    impact.py     impact_of_state(), impact_of_entry(), invalidate_entry(): what relied on it
    diff.py       diff_between(), what_if(): two times, and rows laid over one
    trust.py      agent_trust(): how far the work of an agent held
    snapshot.py   the same four at one time, over the relations that held then
    conversation.py say(), messages(), sessions(), said_about(), said(), summarize(),
                  context(), forget_session(), forget_user()
    recall/
      traverse.py pointing_at(), similar()
      text.py     search(), search_history(), search_entries()
      vector.py   embed(), nearest(), embed_entries(), nearest_entries()
      rank.py     fuse()
    tasks.py      add_task(), claim(), complete(), release()
    values.py     what a name, a key, and a property may hold
  tests/
    devices.py    a test domain that belongs to no product
    oracle.py     an independent oracle for the write rule: random histories and checks
    plain.py      plain implementations of the graph algorithms, to compare with
    test_<module>.py   one per module
    test_attacks.py    what a hostile caller can try
    test_layout.py     the rules below, each proved by a planted violation
    test_readme.py     runs the example in this file

The rules of the layout. tests/test_layout.py checks every one of them, and plants 41 violations in a copy of the package to prove that each check finds what it is for.

1. One module for one capability, named after what it does. No utils, helpers, common, core, base, or manager.
2. No module over 400 lines. A capability that needs more becomes a folder with the same name, split by what each part does.
3. The package imports only the standard library and the database client. A model, an embedder, or a reranker is a function that the application passes in: the package opens no connection of its own, reads no file and no environment variable, loads no code by name, and names no provider.
4. No word of any product in the package.
5. One test file for each module.
6. One database. The Cypher is in the module that sends it: no module only holds queries, and none only forwards them. Nothing is named after a driver or another database.
7. A server (MCP, REST) is a separate project that imports the package. The package has no server, command line, or main module.

Develop

Needs a running FalkorDB (docker compose up -d in the repository root).

uv sync
uv run pytest
uv run ruff check . && uv run ruff format --check .

FALKORDB_HOST and FALKORDB_PORT select the database for the tests. The defaults are 127.0.0.1 and 26379.