recall-livekit

Long-term memory for LiveKit Agents voice agents, backed by Recall by Polign.

The agent loads everything it knows about the caller before the first reply, and the model saves new facts through a remember tool. Facts are typed: each one is a predicate from a closed registry with a value, so "call me Sam" replaces the old name instead of piling up a second one, and the history of the change is kept. No second model runs on the read path, and no embedding service is needed.

pip install recall-livekit "livekit-agents[openai,deepgram,cartesia,silero]"

That is the whole install. pip also brings the polign_db package with the polign CLI and polign-server binaries for Linux, macOS and Windows, so there is no separate database to download, and a worker image needs nothing beyond pip install.

Usage

from livekit.agents import AgentServer, AgentSession, JobContext, JobProcess, cli
from livekit.plugins import cartesia, deepgram, openai, silero
from recall_livekit import VOICE_REGISTRY, RecallAgent, RecallMemory

server = AgentServer()


def setup(proc: JobProcess) -> None:
    # one Recall subprocess per worker process
    proc.userdata["recall"] = RecallMemory.open(
        local_dir="./recall-data",            # the database lives here
        predicates=VOICE_REGISTRY,            # or your own registry file
    )


server.setup_fnc = setup


@server.rtc_session()
async def entrypoint(ctx: JobContext) -> None:
    participant = await ctx.wait_for_participant()
    memory = ctx.proc.userdata["recall"].for_subject(participant.identity)

    session = AgentSession(
        stt=deepgram.STT(), llm=openai.LLM(model="gpt-4.1-mini"),
        tts=cartesia.TTS(), vad=silero.VAD.load(),
    )
    await session.start(
        agent=RecallAgent(
            memory=memory,
            instructions="You are the support line for Acme. Use what you remember about the caller.",
        ),
        room=ctx.room,
    )


if __name__ == "__main__":
    cli.run_app(server)

What happens on a call:

1. on_enter loads the caller's current beliefs and appends them to the instructions inside a <recall_memory> block, before the greeting.
2. The caller says "actually call me Sam, and I moved to Denver". The model calls remember twice. Recall supersedes the old name and timezone, and the agent rewrites its instructions so the next sentence already uses Sam.
3. A week later the same identity calls back and step 1 finds the facts.

Nothing is searched per turn. A caller's typed facts are a short list, so the whole set fits in the prompt. When a caller has more beliefs than limit (default 20), each user turn also runs a search and adds matching facts to that turn only.

Where the memory is stored

local_dir="./recall-data" keeps the database on the worker's machine. The first worker process to open the directory starts a polign-server for it in the background, listening on localhost only, and every other process shares that server. It keeps running between worker restarts; its process id is in recall-data/runtime.json and its log in recall-data/server.log. This is the quickest way to try the package and is fine for a single machine. It works on Linux and macOS.

Workers on several machines need one shared server. Run polign-server somewhere they can all reach (Get started shows how, including storing into S3, GCS or Azure), then connect to it instead:

RecallMemory.open(
    url="http://memory.internal:23000",   # or POLIGN_URL in the environment
    api_key=os.environ["POLIGN_API_KEY"],
    predicates=VOICE_REGISTRY,
)

Your own Agent subclass

from recall_livekit import attach

agent = FrontDesk()                      # any Agent with string instructions
await attach(agent, memory, who="the guest")
await session.start(agent, room=ctx.room)

attach adds the memory block and the remember tool. It does not do the per-turn search for overflowing callers; use RecallAgent for that.

Options

Custom predicates

Predicates are a JSON file. The remember tool's schema is built from it, so the model only ever sees the names you allow.

{
  "name":       { "cardinality": "single", "value_type": "string",  "description": "The name the caller asks to be called" },
  "open_issue": { "cardinality": "multi",  "value_type": "string",  "description": "A problem the caller reported that is not resolved yet" }
}

single means a newer value replaces the old one; multi means each value is an additional fact. value_type is string, number, or boolean. The file replaces Recall's built-in registry entirely. VOICE_REGISTRY is a starter set for callers: name, preferred language, callback number, email, timezone, account tier, recording consent, response style, open issues, owned products, and technologies used.

Subjects and retention

Use a stable, auth-derived identifier as the subject, such as the participant identity your token server issued. Never the room name. Recall keeps the record of every change while the current view updates; enable forget_tool if callers should be able to withdraw a fact by asking, and see the Recall docs for retention and export.

Development

cd python/recall-livekit
python -m pip install -e . pytest pytest-asyncio
pytest tests/unit_tests -q                # fake Recall subprocess, scripted model
pytest tests/integration_tests -v         # real polign-server and polign CLI

The integration tests locate the binaries like the SDK's tests do (POLIGN_SERVER, POLIGN_SOURCE, POLIGN_SERVER_VERSION, or the latest release download) and skip when none is found. To test against the binaries pip installed, set POLIGN_SERVER="$(python -c 'import polign_db; print(polign_db.find_bin("polign-server"))')".