Skip to main content

ZettelForge: Agentic Memory System with vector search, knowledge graph, and synthesis

Project description

ZettelForge

The only agentic memory system built for cyber threat intelligence.

Persistent memory for AI agents and Claude Code — with CTI entity extraction, STIX knowledge graphs, threat-actor alias resolution, and offline-first RAG. MCP server included. No cloud, no API keys.

PyPI Downloads/month Python 3.10+ License: MIT CI

⭐ Star · 📦 pip install zettelforge · 📖 Docs · 🧪 Hosted

Buy Me a Coffee

ZettelForge demo — CTI agentic memory in action

If ZettelForge fits a CTI workflow you run, a star is the fastest signal that this category is worth continuing to invest in.

Why ZettelForge?

General-purpose memory systems don't understand threat intelligence. They can't tell APT28 from Fancy Bear, don't know that CVE-2024-3094 is the XZ Utils backdoor, and can't track how intelligence evolves across reports. When your agent forgets context between investigations, you end up re-reading the same reports and re-building the same mental models.

ZettelForge was built from the ground up for analysts who think in threat graphs, not chat histories. It extracts CVEs, threat actors, IOCs, and MITRE ATT&CK techniques automatically, resolves aliases across naming conventions, builds a knowledge graph with causal relationships, and retrieves memories using intent-aware blended search -- all offline, with no API keys or cloud dependencies.

"Memory augmentation closes 33% of the gap between small and large models on CTI tasks (CTI-REALM, Microsoft 2026)." 1

Feature ZettelForge Mem0 Graphiti Cognee
CTI entity extraction (CVEs, actors, IOCs) Yes No No No
STIX 2.1 ontology Yes No No No
Threat actor alias resolution Yes (APT28 = Fancy Bear) No No No
Knowledge graph with causal triples Yes No Yes Yes
Intent-classified retrieval (5 types) Yes No No No
Offline / local-first (no API keys) Yes No No No
OCSF audit logging Yes No No No
MCP server (Claude Code) Yes No No No

Data Pipeline

ZettelForge architecture — neural recall loop: ingest, enrich, retrieve, synthesize, backed by SQLite + LanceDB

Features

Entity Extraction -- Automatically identifies CVEs, threat actors, IOCs (IPs, domains, hashes, URLs, emails), MITRE ATT&CK techniques, campaigns, intrusion sets, tools, people, locations, and organizations. Regex + LLM NER with STIX 2.1 types throughout.

Knowledge Graph -- Entities become nodes, co-occurrence becomes edges. LLM infers causal triples ("APT28 uses Cobalt Strike"). Temporal edges and supersession track how intelligence evolves.

Alias Resolution -- APT28, Fancy Bear, Sofacy, STRONTIUM all resolve to the same actor node. Works automatically on store and recall.

Blended Retrieval -- Vector similarity (768-dim fastembed, ONNX) + graph traversal (BFS over knowledge graph edges), weighted by intent classification. Five intent types: factual, temporal, relational, exploratory, causal.

Memory Evolution -- With evolve=True, new intel is compared to existing memory. LLM decides ADD, UPDATE, DELETE, or NOOP. Stale intel gets superseded. Contradictions get resolved. Duplicates get skipped.

RAG Synthesis -- Synthesize answers across all stored memories with direct_answer format.

Offline-First -- fastembed (ONNX) for embeddings, llama-cpp-python for LLM features. No API keys, no cloud dependencies.

OCSF Audit Logging -- Every operation is logged in OCSF format (FedRAMP AU controls).

Quick Start

pip install zettelforge
from zettelforge import MemoryManager

mm = MemoryManager()

# Store threat intel -- entities extracted automatically
mm.remember("APT28 uses Cobalt Strike for lateral movement via T1021")

# Recall with alias resolution
results = mm.recall("What tools does Fancy Bear use?")
# Returns the APT28 note (APT28 = Fancy Bear, resolved automatically)

# Synthesize across all memories
answer = mm.synthesize("Summarize known APT28 TTPs")

No TypeDB, no Ollama, no Docker -- just pip install. Embeddings run in-process via fastembed. LLM features (extraction, synthesis) activate when Ollama is available.

With Ollama (enables LLM features)

ollama pull qwen2.5:3b && ollama serve
# ZettelForge auto-detects Ollama for extraction and synthesis

Memory Evolution

# New intel arrives -- evolve=True enables memory evolution:
# LLM extracts facts, compares to existing notes, decides ADD/UPDATE/DELETE/NOOP
mm.remember(
    "APT28 has shifted tactics. They dropped DROPBEAR and now exploit edge devices.",
    domain="cti",
    evolve=True,   # existing APT28 note gets superseded, not duplicated
)

How It Works

Every remember() call triggers a pipeline:

  1. Entity Extraction -- regex + LLM NER identifies CVEs, intrusion sets, threat actors, tools, campaigns, ATT&CK techniques, IOCs (IPv4, domain, URL, MD5/SHA1/SHA256, email), people, locations, organizations, events, activities, and temporal references (19 types)
  2. Knowledge Graph Update -- entities become nodes, co-occurrence becomes edges, LLM infers causal triples
  3. Vector Embedding -- 768-dim fastembed (ONNX, in-process, 7ms/embed) stored in LanceDB
  4. Supersession Check -- entity overlap detection marks stale notes as superseded
  5. Dual-Stream Write -- fast path returns in ~45ms; causal enrichment is deferred to a background worker

Every recall() call blends two retrieval strategies:

  1. Vector similarity -- semantic search over embeddings
  2. Graph traversal -- BFS over knowledge graph edges, scored by hop distance
  3. Intent routing -- query classified as factual/temporal/relational/causal/exploratory, weights adjusted per type
  4. Cross-encoder reranking -- ms-marco-MiniLM reorders final results by relevance

Benchmarks

Evaluated against published academic benchmarks:

Benchmark What it measures Score
CTI Retrieval Attribution, CVE linkage, multi-hop 75.0%
RAGAS Retrieval quality (keyword presence) 78.1%
LOCOMO (ACL 2024) Conversational memory recall 22.0% (with Ollama cloud models)

See the full benchmark report for methodology and analysis.

MCP Server (Claude Code)

Add ZettelForge as a memory backend for Claude Code:

{
  "mcpServers": {
    "zettelforge": {
      "command": "python3",
      "args": ["-m", "zettelforge.mcp"]
    }
  }
}

Your Claude Code agent can now remember and recall threat intelligence across sessions.

Exposed tools: remember, recall, synthesize, entity, graph, stats.

Detection Rules as Memory (Sigma + YARA)

Sigma and YARA rules are first-class memory primitives. Parse, validate, and ingest a rule and its tags become graph edges: MITRE ATT&CK techniques, CVEs, threat-actor aliases, tools, and malware families resolve against the same ontology as every other note. A shared DetectionRule supertype carries SigmaRule and YaraRule subtypes, so a single rule UUID is addressable across both formats.

Sigma rules are validated against the vendored SigmaHQ JSON schema. YARA rules are parsed with plyara and validated against the CCCS YARA metadata standard (tiers: strict, warn, non_cccs). Ingest is idempotent -- re-ingesting an unchanged rule returns the original note via a content-hashed source_ref.

from zettelforge import MemoryManager
from zettelforge.sigma import ingest_rule as ingest_sigma
from zettelforge.yara import ingest_rule as ingest_yara

mm = MemoryManager()
ingest_sigma("rules/proc_creation_win_office_macro.yml", mm)
ingest_yara("rules/webshell_china_chopper.yar", mm, tier="warn")
# Bulk ingest from SigmaHQ or a private rule repo
python -m zettelforge.sigma.ingest /path/to/sigma/rules/
python -m zettelforge.yara.ingest /path/to/yara/rules/ --tier warn

# CI fixture check -- parse + validate, no writes
python -m zettelforge.sigma.ingest rules/ --dry-run

An LLM rule explainer (zettelforge.detection.explainer.explain) produces a structured JSON summary -- intent, key fields, evasion notes, false-positive hypotheses -- for any DetectionRule. It runs synchronously on demand in v1; async enrichment-queue wiring is v1.1. Rate-limited via ZETTELFORGE_EXPLAIN_RPM (default 60 calls/minute).

References: Sigma spec, SigmaHQ rules, CCCS YARA, YARA docs.

Integrations

ATHF (Agentic Threat Hunting Framework)

Ingest completed ATHF hunts into ZettelForge memory. MITRE techniques and IOCs are extracted and linked in the knowledge graph.

python examples/athf_bridge.py /path/to/hunts/
# 12 hunt(s) parsed
# Ingested 12/12 hunts into ZettelForge

See examples/athf_bridge.py.

Extensions

ZettelForge is a complete, production-ready agentic memory system. Everything documented above works out of the box.

For teams that need TypeDB-scale graph storage, OpenCTI integration, or multi-tenant deployment, optional extensions are available:

Extension What it adds
TypeDB STIX 2.1 backend Schema-enforced ontology with inference rules
OpenCTI sync Bi-directional sync with OpenCTI instances
Multi-tenant auth OAuth/JWT with per-tenant isolation
Sigma rule generation Detection rules from extracted IOCs

Extensions are installed separately:

pip install zettelforge-enterprise

Hosted option: ThreatRecall provides managed ZettelForge with all extensions, so you don't have to run infrastructure yourself.

Configuration

Variable Default Description
AMEM_DATA_DIR ~/.amem Data directory
ZETTELFORGE_BACKEND sqlite SQLite community backend. TypeDB is available via extension.
ZETTELFORGE_LLM_PROVIDER local local (llama-cpp) or ollama

See config.default.yaml for all options.

Contributing

See CONTRIBUTING.md for development setup.

License

MIT -- See LICENSE.

Made by Patrick Roland.

Support the Project

ZettelForge is MIT-licensed. If it's useful in your workflow and you'd like to help keep it maintained:

Buy Me a Coffee

Acknowledgments

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zettelforge-2.4.0.tar.gz (9.6 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

zettelforge-2.4.0-py3-none-any.whl (172.8 kB view details)

Uploaded Python 3

File details

Details for the file zettelforge-2.4.0.tar.gz.

File metadata

  • Download URL: zettelforge-2.4.0.tar.gz
  • Upload date:
  • Size: 9.6 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for zettelforge-2.4.0.tar.gz
Algorithm Hash digest
SHA256 e4694ccc386054871c1948913203121f9d3ab50313cf815663f299876b7a4a24
MD5 1972803746214b186ea271f03b60ca8b
BLAKE2b-256 95ee1fed24e759f8f4440b0c2329b8f4e695a7b2229f666bc8490dcabfbcb3cf

See more details on using hashes here.

Provenance

The following attestation bundles were made for zettelforge-2.4.0.tar.gz:

Publisher: publish.yml on rolandpg/zettelforge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file zettelforge-2.4.0-py3-none-any.whl.

File metadata

  • Download URL: zettelforge-2.4.0-py3-none-any.whl
  • Upload date:
  • Size: 172.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for zettelforge-2.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 5385de0879533dc4398e806043cd975e42045e12432f0ae89e0599b422f7ba7b
MD5 0bf9ff06301e259332ecf917f8c3fd7e
BLAKE2b-256 c6bd3c822fde977dabf99d50ec23a179b5be27cd645920dd5617ce167d1db5fb

See more details on using hashes here.

Provenance

The following attestation bundles were made for zettelforge-2.4.0-py3-none-any.whl:

Publisher: publish.yml on rolandpg/zettelforge

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page