LearnKit

🚀 Live Pre-Release on PyPI! LearnKit is installable via PyPI as learnkit-ai. It provides the complete experience-distillation layer for Python AI agents. Let your agents compound knowledge dynamically!

Fine-Tuning Without Fine-Tuning

LearnKit is an agent-agnostic SDK that gives any AI agent a self-improving memory layer.

Most agents today suffer from amnesia or rely on naive memory (storing endless raw chat logs). This creates:

• “Memory soup”
• Exploding context windows
• No signal on whether a past action was actually successful

LearnKit replaces raw chat logs with Experience Distillation.

Every time your agent runs, LearnKit:

1. Evaluates the execution trace
2. Extracts what worked (and what failed)
3. Compiles reusable structured memory artifacts

Over time, the agent builds a compounding “wiki” of expertise — without retraining the underlying model.

Core Philosophy

LearnKit treats agent memory like a curated wiki operating across three continuous loops:

1. Ingest (The Distiller)

After a task completes, LearnKit analyzes the agent’s Chain-of-Thought (CoT).

• Successful traces → distilled into reusable SkillRecord
• Failed traces → converted into FailureRecord
• Prevents agents from repeating known mistakes

2. Query (The Retriever)

Before a task begins:

• LearnKit classifies the domain and task type
• Retrieves high-confidence relevant memories
• Injects only the most useful context

3. Maintain (The Evolver)

Memory is continuously optimized:

• Unused records decay over time
• High-value skills evolve automatically
• GEPA-based prompt mutation discovers better strategies

To install from PyPI (recommended):

pip install learnkit-ai
# Or with integration extras:
pip install "learnkit-ai[langchain]"

To install from local repo root:

pip install -e .                    # core SDK
pip install -e ".[langchain]"       # adds LangChain + langchain-anthropic
pip install -e ".[dev]"             # pytest + pytest-asyncio

Other optional extras: mem0, zep, qdrant.

Set your Anthropic key once (PowerShell, persists across sessions):

[Environment]::SetEnvironmentVariable("ANTHROPIC_API_KEY", "sk-ant-...", "User")

On bash/zsh: export ANTHROPIC_API_KEY=sk-ant-... in your shell rc.

60-second Quick Start

python examples/quick_start.py

Walks through 5 parts that exercise the whole SDK:

Wrap your agent — 5 lines

import learnkit as lk

memory = lk.LearnKit(memory_backend="sqlite", scope="user")

@memory.agent(domain="coding")
def my_agent(task: str, _learnkit_context: str = "") -> str:
    # _learnkit_context is injected by the decorator on every call.
    # Splice it into your prompt however your framework expects.
    return call_your_llm(prompt=task, system=_learnkit_context)

# Same task, called twice — run 2 sees what run 1 distilled.
my_agent("Debug a Python multiprocessing deadlock on macOS")
my_agent("Debug a Python multiprocessing deadlock on macOS")

Valid scope values: "user", "team", "public" (see learnkit/schemas/base.py).

Integrate with LangChain

A runnable end-to-end demo lives at examples/langchain_demo.py. It wraps a real LangChain 1.x tool-calling agent (create_agent + ChatAnthropic + two tools) with @memory.agent, then runs the same task twice against a file-backed SQLite store:

RUN 1 (cold memory):    [LearnKit] Context injected:   0 chars
RUN 2 (warm memory):    [LearnKit] Context injected: 610 chars

Run 2's answer is qualitatively richer because the skill, facts, and failures distilled from run 1's trajectory get retrieved and spliced into the system prompt. The demo uses background_postprocess=False so distillation runs synchronously and the second call is guaranteed to see the first call's output — drop that flag for production.

To run it yourself:

pip install -e ".[langchain]"
python examples/langchain_demo.py

How it works — the 8-step loop

The agent function never changes. The decorator orchestrates everything around it.

1. User calls your wrapped agent with a task.
2. Classify — TaskClassifier returns a domain vector, e.g. {"Python": 0.9, "Concurrency": 0.7}.
3. Retrieve — SemanticRetriever pulls relevant records (FTS5 lexical + optional dense rerank), filtered by domain and scope.
4. Compose — compose_context formats records into a bounded prompt block (≤ 8 records, ≤ 1,200 tokens, inference mode = PRESCRIPTIVE / GUIDED / EXPLORATORY based on top-record confidence).
5. Run — your function executes with _learnkit_context injected as a kwarg.
6. Evaluate — Evaluator.evaluate_with_llm_judge scores the response 0–5.
7. Distill — if score ≥ quality_threshold (default 3.5), MemoryDistiller emits new SkillRecord / FactRecord / FailureRecord / TraceRecord. Below threshold, a FailureRecord is stored directly so future runs avoid the same path.
8. Persist — records are written via the active backend; the trajectory is registered against a per-run ID for inspection.

Memory model

LearnKit stores seven typed record kinds (learnkit/schemas/):

Bounded memory is enforced at retrieval: the router caps results at 8 records / ~1,200 tokens before the composer formats them.

Maintenance

Call memory.maintain_memory() periodically (cron, background job, etc.):

memory.maintain_memory(weeks=1, decay_rate=0.02, quarantine_hours=24)
# → {"decayed": N, "stale": M, "promoted": K}

• Decay: every active/stale record loses decay_rate confidence per weeks elapsed.
• Stale: records past expires_at get marked stale and excluded from retrieval.
• Promote: quarantined records older than quarantine_hours are promoted to active.

Architecture & contributing

Run the test suite:

pytest tests/ -q       # 48 passing, ~1s

Pre-commit hooks (black / ruff / isort / whitespace / yaml / debug-statements) are enforced on commit:

pip install pre-commit
pre-commit install

Status

v0.0.2 — Live Pre-Release. The full ingest / query / maintain loop runs end-to-end with SQLite + FTS5 + DSPy classifier + LLM-judge evaluator + structured distiller. Published and installable from PyPI as learnkit-ai! See AGENTS_V2.md for the production hardening plan and improvements.md for the open backlog.