zer0lint 0.2.1

AI memory extraction diagnostics — works with mem0 or any HTTP memory endpoint. By Hermes Labs.

zer0lint

Memory extraction diagnostics for mem0 configs and HTTP memory endpoints. Silent extraction failure is an agent-security risk: your agent drops facts, nothing errors, behavior drifts. zer0lint measures it and writes a validated fix to the correct config field.

---

30-second pitch

AI agents using mem0 (or any HTTP memory endpoint) can silently lose facts during the LLM extraction step. add() returns success, search() returns results, retrieval benchmarks look green — but the specifics never landed. Attackers and red-teamers exploit this gap; operators miss it entirely because there's no error.

zer0lint check injects 5 known technical facts into your live memory system and measures round-trip recall. zer0lint generate writes a validated extraction prompt to the correct config field (custom_fact_extraction_prompt) — the field that actually controls extraction in mem0 v1.x, which most developers get wrong.

pip install zer0lint
zer0lint check --config ~/.mem0/config.json
# Score  : 0/5 (0%) — CRITICAL
zer0lint generate --config ~/.mem0/config.json
# Before : 0/5 (0%)   After : 5/5 (100%)   Δ : +100pp

---

Is this you?

• You set up mem0 and your AI agent still forgets things it should remember
• add() returns success, search() returns results, but the agent can't recall specific facts
• You ran a retrieval benchmark — numbers look fine — but the agent still seems broken
• You switched LLM models and memory got worse, not better
• You have no idea if your extraction pipeline is actually working

If any of these match: your extraction is probably broken and completely silent. zer0lint diagnoses it in under a minute.

pip install zer0lint
zer0lint check --config ~/.mem0/config.json

---

The Problem

The failure is invisible. add() returns {"results": [...]}. search() returns results. Retrieval benchmarks show 90%+ hit@any. But when the LLM extraction step produces malformed JSON or drops specifics, the facts never land — degraded fallbacks get stored instead. You won't see an error. You'll just notice your agent doesn't remember.

Proof from a real run (2026-03-22, mistral:7b, default mem0 config):

Score  : 0/5 (0%) — CRITICAL
  ⚠  Model upgrade: We switched from gpt-3.5-turbo to gpt-4o-mini...
  ⚠  API endpoint: The API service runs on port 8421 with TLS 1.3...
  ⚠  CI status: CI pipeline passed on 2026-03-22 at commit a3f8c12...
  ⚠  Configuration: Auth tokens expire after 3600 seconds...
  ⚠  Version update: Updated Redis cluster to v7.2.4...

After running zer0lint generate (same model, same config, new extraction prompt):

Score  : 5/5 (100%) — HEALTHY
  Δ    : +100pp

Same model. One config change. 0%→100%.

---

Quick Start

pip install zer0lint

# Step 1: diagnose
zer0lint check --config ~/.mem0/config.json

# Step 2: fix (if score < 80%)
zer0lint generate --config ~/.mem0/config.json

# Dry run first if you want to see what changes before applying
zer0lint generate --config ~/.mem0/config.json --dry-run

Your original config is always backed up before any changes are written.

Universal HTTP mode

Not using mem0? zer0lint works with any memory system that exposes add/search over HTTP:

# Point at any memory server — no mem0 dependency needed
zer0lint check --add-url http://localhost:19420/add --search-url http://localhost:19420/recall_b

# Generate and save the extraction prompt for your system
zer0lint generate --add-url http://localhost:19420/add --search-url http://localhost:19420/recall_b --save-prompt prompt.txt

Works with cogito-ergo, Zep, LangMem, or any custom HTTP memory API.

---

What It Does

zer0lint check

Injects 5 synthetic technical facts into your live mem0 instance, then measures round-trip recall. Uses your existing LLM — no new API keys or models required.

zer0lint v0.2.1 — extraction health check
Config : ~/.mem0/config.json
Model  : mistral:7b
Prompt : default (mem0 built-in)

Error in new_retrieved_facts: Unterminated string starting at: line 1 column 10 (char 9)
Error in new_retrieved_facts: Expecting ',' delimiter: line 1 column 13 (char 12)

[CHECK] Using model: mistral:7b
[CHECK] Testing with 5 synthetic facts...
[CHECK] Score: 0/5 (0%) — CRITICAL
  ⚠  Model upgrade: We switched from gpt-3.5-turbo to gpt-4o-mini...
  ⚠  API endpoint: The API service runs on port 8421 with TLS 1.3...
  ⚠  CI status: CI pipeline passed on 2026-03-22 at commit a3f8c12...
  ⚠  Configuration: Auth tokens expire after 3600 seconds...
  ⚠  Version update: Updated Redis cluster to v7.2.4...

Score  : 0/5 (0%) — CRITICAL
Run zer0lint generate to diagnose and fix.

Statuses: HEALTHY (≥80%) · ACCEPTABLE (60–79%) · DEGRADED (40–59%) · CRITICAL (<40%)

zer0lint generate

3-phase diagnostic + fix. Validates the prompt works before applying it. Never writes without proof of improvement.

1. Baseline — test your current config as-is
2. Re-test — apply zer0lint's domain-aware extraction prompt at config level
3. Apply — if improved, write the validated prompt to your config (with backup)

[1/3] Baseline — testing current config as-is...
  Baseline score: 0/5 (0%)
    ❌ Configuration
    ❌ API endpoint
    ❌ CI status
    ❌ Model upgrade
    ❌ Version update

[2/3] Re-testing with zer0lint technical extraction prompt (config-level)...
  Improved score: 5/5 (100%)
  Improvement: +100pp
    ✅ Configuration
    ✅ API endpoint
    ✅ CI status
    ✅ Model upgrade
    ✅ Version update

[3/3] Applying fix to config (0% → 100%)...
  ✅ Config updated.
  Backup at: ~/.mem0/config.backup.2026-03-22T02:18:34.json

Results:
  Before : 0/5 (0%)
  After  : 5/5 (100%)
  Δ      : +100pp

---

Critical Discovery: Where Extraction Actually Happens

Most developers who hit this problem try to fix it by passing a custom prompt at call time:

memory.add("...", prompt="extract technical facts")  # does nothing

This has no effect in mem0 v1.x. The extraction prompt must live in the config — specifically in the custom_fact_extraction_prompt field. There is no error when you pass it to add(). It simply has zero effect on what gets extracted.

zer0lint writes the validated prompt to the correct location. That's the fix.

---

Config Format

zer0lint reads a standard mem0 config JSON. Example:

{
  "llm": {
    "provider": "ollama",
    "config": {
      "model": "mistral:7b",
      "ollama_base_url": "http://localhost:11434"
    }
  },
  "vector_store": {
    "provider": "chroma",
    "config": {
      "collection_name": "my_agent_memory",
      "path": "~/.mem0/chroma"
    }
  }
}

After zer0lint generate, it adds:

{
  "custom_fact_extraction_prompt": "You are a Technical Memory Organizer..."
}

If you're using cogito-ergo, your config lives at ~/.cogito/config.json — same format. Or skip the config entirely and use HTTP mode with cogito-ergo's endpoints.

---

Test Results (2026-03-22)

Model comparison, 5 technical facts:

Model	Default prompt	zer0lint prompt	Δ
qwen3.5:4b	80%	100%	+20pp
mistral:7b	0%	100%	+100pp

Scale test, 10 facts across 5 domains:

	Score	%
Default	7/10	70%
zer0lint	9/10	90%

mistral:7b with default mem0 prompt produces malformed JSON — Unterminated string, Expecting ',' delimiter — and silently drops facts. zer0lint's extraction prompt fixes this completely on the same model with no other changes.

Improvement varies by model. Smaller models that struggle with structured JSON output see the largest gains.

---

How It Works

zer0lint borrows the LLM you already have configured in your mem0 config. No new API keys, no new models, no cloud calls beyond what you already use.

It injects known facts, measures how many survive the extraction round-trip, generates a prompt that improves the score, validates the improvement, then writes to config. Your original is backed up with an ISO timestamp before anything is changed.

---

Installation

# From PyPI (recommended)
pip install zer0lint

# From source
git clone https://github.com/roli-lpci/zer0lint
cd zer0lint
pip install -e .

Requirements: Python 3.9+. For mem0 config mode: pip install zer0lint[mem0]. For HTTP mode: no extra dependencies.

---

Built by Hermes Labs

zer0lint is part of the Hermes Labs AI agent tooling suite:

• lintlang — Static linter for AI agent tool descriptions and prompts
• Little Canary — Prompt injection detection
• Suy Sideguy — Runtime policy enforcement for agents
• zer0lint — Memory extraction diagnostics ← you are here

---

Supported Systems

zer0lint is architected to work over HTTP with any memory system that exposes add/search endpoints. It should work for most agent memory setups if configured correctly.

System	Status	How
mem0 v1.x	✅	--config flag
cogito-ergo	✅	--add-url + --search-url
Agent Gorgon	✅	--add-url + --search-url
Any HTTP memory API	✅	--add-url + --search-url

---

When to use it

• You run mem0 (or any HTTP memory endpoint) in production and facts are going missing
• You switched LLM models and memory quality dropped
• You want a CI health check for memory extraction (exit code non-zero on DEGRADED/CRITICAL)
• You're red-teaming or auditing an agent stack and need a diagnostic for silent extraction loss

When not to use it

• You're not using mem0 or any HTTP memory endpoint — this is a diagnostic, not a memory system
• Your extraction score is already ≥80% (HEALTHY) on zer0lint check
• The failure is a connection error (mem0 can't reach the LLM/vector store) — that's upstream, not extraction

---

License

Apache 2.0

---

Built by Hermes Labs - Rolando Bosch (@roli-lpci)