llm-kelt

Knowledge · Embedding · Learning · Training

Persistent memory and fine-tuning data for LLM applications, backed by Postgres.

Store the things an LLM needs to know or learn from — facts, feedback, preferences, predictions, directives — under an isolation key. Retrieve them for prompt injection or RAG. Export them to DPO/SFT/classifier datasets. Train LoRA/DPO/Prompt adapters from the exported data.

Requirements

• Python 3.11+
• PostgreSQL 16+ with the vector extension (pgvector)
• For training: CUDA GPU (or MPS on Apple Silicon)

Install

pip install llm-kelt              # runtime
pip install llm-kelt[training]    # + torch / transformers / peft / trl

Minimal example

from appinfra.config import Config
from appinfra.log import LogConfig, LoggerFactory
from llm_kelt import ClientContext, ClientFactory
from llm_kelt.inference import ContextBuilder

config = Config("etc/llm-kelt.yaml")
lg = LoggerFactory.create_root(LogConfig.from_params(level="warning"))

kelt = ClientFactory(lg).create_from_config(
    context=ClientContext(context_key="my-agent"),
    config=config,
)

kelt.atomic.assertions.add("Timezone: UTC", category="settings")
kelt.atomic.assertions.add("Prefers concise, code-first answers", category="style")

system_prompt = ContextBuilder(kelt.atomic.assertions).build_system_prompt(
    base_prompt="You are a helpful assistant.",
)
# → "You are a helpful assistant.\n\n## About the user:\n- Timezone: UTC\n- ..."

That's the whole shape: put things in under a context_key, pull them back out grouped for prompt injection. Everything else (RAG, feedback, preferences, training) builds on the same model.

Where to go next

• Quickstart — 5 minutes from install to first RAG query.
• Concepts — context keys, schemas, atomic vs KG. Read once before the tutorials.
• Atomic memory — the seven fact clients (assertions, feedback, preferences, predictions, directives, interactions, solutions) and how they relate.
• Context & RAG — embedding facts, semantic search, ContextQuery.
• Conversation — multi-turn sessions, token accounting, compaction, storage.
• Training — manifest workflow, LoRA/DPO/SFT/Prompt, exports, adapter registry.
• Knowledge graph — entities, aliases, hierarchical scopes.
• Multi-schema — SchemaMode, with_schema(), isolation.
• CLI reference — kelt atomic|proxy|train|session.
• Glossary — project-specific terms.

Configuration

The library reads its config from etc/llm-kelt.yaml. Key sections:

dbs:
  main:
    url: postgresql://user:pass@localhost:5432/llm_kelt
    extensions: [vector]

llm:
  default_backend: local
  backends:
    local:
      base_url: http://localhost:8000/v1
      model: default

embedding:
  type: openai
  base_url: http://localhost:8001/v1
  model: text-embedding-3-small

kelt:
  adapters:
    lora:
      base_path: ~/models/adapters

llm, embedding, and kelt.adapters are only required for the subsystems that use them (ContextQuery, RAG, and training respectively).

Examples

Runnable scripts in examples/:

• 01_facts_and_context.py — assertions + ContextBuilder.
• 02_rag_retrieval.py — embeddings, search_similar, ContextQuery with RAG.
• 03_training_export.py — feedback + preferences → DPO/SFT/classifier JSONL.
• 04_lora_training.py — end-to-end LoRA training.
• 05_conversation.py — Conversation, compaction, FileSessionStorage.

License

Apache 2.0

Maintained by LLM Works LLC and contributors.