SmartMemory — Typed AI Memory for LLM Agents
Read the docs · Compare vs Mem0 / Zep / Letta / claude-mem · Maya sample app
SmartMemory is an AI memory system with five core typed memory types — episodic, semantic, procedural, pending (short-term, formerly "working"), and zettelkasten — instead of a single vector store. It combines a knowledge graph (FalkorDB), vector embeddings, and an observable 11-stage extraction pipeline so LLM agents can remember, reason over relationships, and update structured knowledge across sessions and tools.
Who this is for
- Pick SmartMemory when you need typed memory that drops into LangChain, LlamaIndex, the Anthropic Agent SDK, or any MCP client; when you want bitemporal accuracy (event time + system time); when multi-tenant isolation must work out of the box; when you want to inspect every stage of extraction.
- Pick something else when you only need a personalization shim for chatbot history (Mem0 is simpler), a complete agent framework (Letta), or zero-config session capture for Claude Code (claude-mem). See the full comparison.
What makes it different
- 5 core typed memory types (not a single blob/vector): episodic events, semantic facts, procedural skills, pending/short-term context, zettelkasten links — plus an expertise layer (decision, constraint, learned, opinion, reasoning, observation).
- Knowledge graph by default, with multi-hop and semantic-hop retrieval — query relationships, not just similarity.
- Bitemporal time model preserved at the storage layer — every fact has both an event time and a system time.
- Observable 11-stage pipeline: classify → coreference → simplify → entity_ruler → llm_extract → ontology_constrain → store → link → enrich → ground → evolve. Per-stage timing, tokens, cost.
- Native MCP server, multi-tenant workspaces, and origin-provenance tiers built in — not bolted on.
- Expertise layer — first-class
decision(withrejected_alternatives,rationale,constraints),constraint,learned,opinion,reasoning,observationmemory types. Each has its own Manager + Queries with supersession and reinforcement. Ergonomic capture viamem.add_decision(),mem.add_constraint(),mem.add_learning(),mem.add_opinion(),mem.add_reasoning(),mem.add_observation(). Recall partitioned by type viamem.search(query, expertise=True).
Expertise vs knowledge
SmartMemory's memory types fall into two cohorts. The knowledge layer (semantic, episodic, procedural, zettel) stores what's true. The expertise layer (decision, constraint, learned, opinion, reasoning, observation) stores what to do — and what not to do. The expertise layer is what makes the substrate useful for acting, not just recalling: captured choices, rejected alternatives, hard constraints, lessons learned. See Expertise vs Knowledge for the full mapping and recall channel.
🚀 Quick Install
pip install smartmemory # Everything: local memory + MCP server + graph viewer + CLI
pip install smartmemory-core[lite] # Core library only, local mode (for developers)
pip install smartmemory-core[server] # Core library only, server mode (FalkorDB + Redis)
smartmemoryis the distribution package. A single install bundlessmartmemory-core[lite](local SQLite + usearch storage), the unified MCP server, the graph viewer, and the CLI — there is no separate[local]extra. You pick local or remote mode atsmartmemory setuptime, not at install time.smartmemory-coreis the core library for developers building on top of SmartMemory.
SmartMemory Lite — No Docker Required
from smartmemory.tools.factory import create_lite_memory, lite_context
# Simple usage — full LLM extraction runs if OPENAI_API_KEY is set
memory = create_lite_memory()
item_id = memory.ingest("Alice leads Project Atlas")
results = memory.search("who leads Atlas", top_k=5)
# Preferred in scripts — cleans up globals and closes SQLite on exit
with lite_context() as memory:
item_id = memory.ingest("Alice leads Project Atlas")
results = memory.search("who leads Atlas")
# Force no LLM calls (even if OPENAI_API_KEY is set)
from smartmemory.pipeline.config import PipelineConfig
memory = create_lite_memory(pipeline_profile=PipelineConfig.lite(llm_enabled=False))
Or via CLI:
smartmemory-core add "Alice leads Project Atlas"
smartmemory-core search "who leads Atlas"
smartmemory-core rebuild # Reindex vector store from graph data
smartmemory-core watch /path/to/vault # Auto-ingest new/changed .md files
Architecture Overview
SmartMemory implements a multi-layered memory architecture with the following components:
Core Components
- SmartMemory: Main unified memory interface (
smartmemory.smart_memory.SmartMemory) - SmartGraph: Graph database backend using FalkorDB for relationship storage
- Memory Types: Specialized memory stores for different data types
- Pipeline Stages: Processing stages for ingestion, enrichment, and evolution
- Plugin System: Extensible architecture for custom evolvers and enrichers
Memory Types
- Pending Memory: Short-term buffer for items awaiting consolidation (formerly "working"; routed at ingest by the
ConsolidationRouter) - Semantic Memory: Facts and concepts with vector embeddings
- Episodic Memory: Personal experiences and learning history
- Procedural Memory: Skills, strategies, and learned patterns
- Zettelkasten Memory: Bidirectional note-taking system with AI-powered knowledge discovery
- Reasoning Memory: Chain-of-thought traces capturing "why" decisions were made (System 2)
- Opinion Memory: Beliefs with confidence scores, reinforced or contradicted over time
- Observation Memory: Synthesized entity summaries from scattered facts
- Decision Memory: First-class decisions with confidence tracking, provenance chains, and lifecycle management
Storage Backends
- Lite mode: SQLite graph + usearch vectors — no Docker, no external services
- Server mode: FalkorDB (graph + vectors) + Redis (caching) — full-featured, requires Docker
Processing Pipeline
ingest() runs an 11-stage pipeline:
classify → coreference → simplify → entity_ruler → llm_extract → ontology_constrain → store → link → enrich → ground → evolve
Each stage implements the StageCommand protocol (execute(state, config) → state, undo(state) → state). The pipeline supports breakpoint execution (run_to(), run_from(), undo_to()) for debugging and resumption.
add() is simple storage: normalize → store → embed (use for internal/derived items).
Key Features
- 11 Curated Memory Types: Pending, Semantic, Episodic, Procedural, Zettelkasten, Reasoning, Opinion, Observation, Decision, Constraint, Learned (plus structural types like
code,plan,evaluation,anchor,tool_callused internally) - 11-Stage NLP Pipeline: classify → coreference → simplify → entity_ruler → llm_extract → ontology_constrain → store → link → enrich → ground → evolve
- Self-Learning EntityRuler: Pattern-matching NER that improves with use — LLM discoveries feed back into rules
- Evolver Framework: Core auto-registered evolvers plus specialist lifecycle evolvers for decay, consolidation, opinion synthesis, retrieval-based strengthening, Hebbian co-retrieval, and stale memory detection
- Code Indexer: AST-based Python + TypeScript parser with cross-file call resolution, semantic code search, and memory↔code graph bridging
- Zero-Infra Lite Mode: SQLite + usearch backend —
pip install smartmemory-core[lite]and go - Server Mode: FalkorDB graph + Redis caching for production-scale deployments
- Hybrid Search: Graph-structured search + BM25/embedding RRF fusion with query decomposition for compound queries
- Auditable Memory: Tamper-evident hash chain over every item's version history.
explain()answers "how do you know that?" in one call (origin, version audit, supersession lineage, chain verification), and search acceptsas_of_datefor transaction-time travel - Auto-Registered Plugins: 6 enrichers, 10 default evolvers, 1 grounder, and 3–5 extractors (3 always loaded + spaCy and GLiNER2 when their optional deps are installed) — plus one lazy catalog of 29 evolvers selectable in Studio and additional specialist plugins for opt-in use
- Plugin Security: Sandboxing, permissions, and resource limits for safe plugin execution
- Flexible Scoping: Optional
ScopeProviderfor multi-tenancy or unrestricted OSS usage
📦 Installation
From PyPI (Recommended)
# Lite mode (zero infra — SQLite + usearch, no Docker required)
# Embeds with all-MiniLM-L6-v2 on ONNX Runtime: ~150 MB, sub-second cold start, no torch.
pip install smartmemory-core[lite]
# Server mode (FalkorDB + Redis, requires Docker or manual install; includes the torch stack)
pip install smartmemory-core[server]
# Embedding runtime (same model, same vectors — pick by hardware)
pip install smartmemory-core[torch] # sentence-transformers on CPU (previous default)
pip install smartmemory-core[gpu] # sentence-transformers; CUDA used automatically when visible
pip install smartmemory-core[rerank] # Cross-encoder reranker (needs torch)
# Optional features
pip install smartmemory-core[cli] # CLI tools
pip install smartmemory-core[watch] # Vault watcher for Markdown files
pip install smartmemory-core[wikipedia] # Wikipedia enrichment
pip install "smartmemory-core[coreference]" # Coreference: resolves he/she/it on save, downloads a model (optional)
pip install smartmemory-core[all] # All optional features
Coreference is optional and off unless you install the coreference extra (it pulls fastcoref). It resolves he/she/it on save and downloads a model on first use.
Models and hermetic startup
Entity extraction needs a spaCy model, and local embedding needs the MiniLM weights. The
library never installs anything at import time: create_lite_memory() raises
MissingModelError naming the exact command when a model is absent. Pass
auto_download_models=True to let it fetch them (the smartmemory CLI does this during
smartmemory setup).
python -m spacy download en_core_web_sm
SMARTMEMORY_HF_ALLOW_DOWNLOAD=1 python -c "from smartmemory.plugins.embedding import EmbeddingService; EmbeddingService().embed('warm')"
The embedding backend is auto-detected at first use: torch with CUDA, then torch on CPU, then
ONNX Runtime. Vectors are identical across backends, so a store written by one opens with any
other. Force one with SMARTMEMORY_EMBEDDING_BACKEND=onnx|torch|auto.
Update check
The first time a SmartMemory() is constructed in a process, the runtime asks
https://api.smartmemory.ai/memory/lite/update-check once whether a newer release is
published. If there is one, it logs a single line on the smartmemory.update_check
logger. Nothing is printed to stdout and nothing blocks: the request runs on a daemon
thread with a 2 second timeout. Operational failures only log a warning naming what
was lost; they never interrupt construction.
The request carries exactly these fields:
- an anonymous install id, a random uuid4 generated on first run and stored at
~/.smartmemory/install_id(or under$SMARTMEMORY_HOME) - the SmartMemory version, the Python version,
sys.platform, and the CPU architecture - whether the backend is lite or server
- your redistribution license key, only if you set
SMARTMEMORY_LICENSE_KEY(this update check is the only code that reads that variable: it gates nothing and is never read when the check is disabled)
No memory content, no file paths, no hostname, no username, and no IP-derived identity are sent. The install id is random and is never derived from your hardware. At most one request is made per 24 hours per install.
To turn it off, set either SMARTMEMORY_NO_UPDATE_CHECK=1 or the cross-tool
DO_NOT_TRACK=1. Setting SMARTMEMORY_UPDATE_CHECK_URL to an empty string also
disables it, and the check is skipped automatically when a CI environment variable is
present.
See the licensing documentation for redistribution terms and the update-check disclosure.
Infrastructure
Lite mode (smartmemory-core[lite]): No external services needed. SQLite and usearch are bundled.
Server mode (smartmemory-core[server]): Requires FalkorDB and Redis:
# Docker Compose (recommended) — from repository root
docker-compose up -d
# Starts FalkorDB on port 9010, Redis on port 9012
# Or manually
docker run -d -p 9010:6379 falkordb/falkordb:6.0.3
docker run -d -p 9012:6379 redis:7-alpine
# Verify
redis-cli -p 9010 PING # FalkorDB
redis-cli -p 9012 PING # Redis
🎯 Quick Start
Basic Usage (Lite Mode)
from smartmemory.tools.factory import create_lite_memory
# No Docker, no config — just works
memory = create_lite_memory()
# Ingest a memory (full pipeline: extract → store → link → enrich → evolve)
item_id = memory.ingest("User prefers Python for data analysis tasks")
# Or use add() for simple storage without pipeline
item = MemoryItem(
content="Quick note about Python",
memory_type="semantic",
metadata={'topic': 'preferences'}
)
memory.add(item)
# Search memories (automatically scoped)
results = memory.search("Python programming", top_k=5)
for result in results:
print(f"Content: {result.content}")
print(f"Type: {result.memory_type}")
# Get memory summary
summary = memory.get_all_items_debug()
print(f"Total memories: {summary['total_items']}")
Conflict evidence in core reads
The unreleased RECALL-CONFLICT-FLAG-1 core slice adds a response-only item.conflict field to search results. It names live peers with supported disagreements in the returned pool. Conflicting peers sort newest first, with stable IDs breaking ties. The top-level conflict key is reserved for read responses and is discarded when rebuilding an item from a result dictionary. A null field does not prove consistency, especially when a small top_k excludes a peer.
from smartmemory.errors import RecallConflictError
try:
results = memory.search("Acme retries", fail_if_conflicted=True)
except RecallConflictError as error:
print(error.pairs) # Scoped ID pairs, without claim text in the error message
recall_pack(..., fail_if_conflicted=True) checks admitted gathered memory candidates before budgeting. Its conflict flag identifies a conflicted pack, its manifest records pairs and its warning labels count inside the token budget. Section priority remains unchanged. API, SDK and MCP exposure is pending. The core contract is smart-memory-docs/docs/features/RECALL-CONFLICT-FLAG-1/recall-conflict-contract.json on the corresponding local author branch.
Using Different Memory Types
from smartmemory import SmartMemory, MemoryItem
# Initialize SmartMemory
memory = SmartMemory()
# Add pending memory (short-term context awaiting consolidation)
pending_item = MemoryItem(
content="Current conversation context",
memory_type="pending"
)
memory.add(pending_item)
# Add semantic memory (facts and concepts)
semantic_item = MemoryItem(
content="Python is a high-level programming language",
memory_type="semantic"
)
memory.add(semantic_item)
# Add episodic memory (experiences)
episodic_item = MemoryItem(
content="User completed Python tutorial on 2024-01-15",
memory_type="episodic"
)
memory.add(episodic_item)
# Add procedural memory (skills and procedures)
procedural_item = MemoryItem(
content="To sort a list in Python: use list.sort() or sorted(list)",
memory_type="procedural"
)
memory.add(procedural_item)
# Add Zettelkasten note (interconnected knowledge)
zettel_item = MemoryItem(
content="# Machine Learning\n\nML learns from data using algorithms.",
memory_type="zettel",
metadata={'title': 'ML Basics', 'tags': ['ai', 'ml']}
)
memory.add(zettel_item)
CLI Usage (Optional)
# Install CLI tools
pip install smartmemory-core[cli]
# Add a memory
smartmemory-core add "Python is great for AI"
# Add without creating a markdown file
smartmemory-core add "Quick note" --no-markdown
# Search memories
smartmemory-core search "Python programming" --top-k 5
# Search with JSON output
smartmemory-core search "Python programming" --json
# Rebuild the vector index from graph data
smartmemory-core rebuild
# Auto-ingest new/changed Markdown files from a vault directory
pip install smartmemory-core[watch]
smartmemory-core watch /path/to/vault
Use Cases
Conversational AI Systems
- Maintain context across multiple conversation sessions
- Learn user preferences and adapt responses
- Build comprehensive user profiles over time
Educational Applications
- Track learning progress and adapt teaching strategies
- Remember previous topics and build upon them
- Personalize content based on individual learning patterns
Knowledge Management
- Store and retrieve complex information relationships
- Connect related concepts across different domains
- Evolve understanding through continuous learning
- Build a personal knowledge base with Zettelkasten method
Personal AI Assistants
- Remember user preferences and past interactions
- Provide contextually relevant recommendations
- Learn from user feedback to improve responses
Examples
The examples/ directory contains a broader set of demonstration scripts than the highlights below. See examples/README.md for the full catalog and setup notes.
memory_system_usage_example.py: Basic memory operations (ingest, search, delete)factory_usage_example.py: Factory helpers for creating memory and store instancespipeline_v2_example.py: Breakpoints, resumption, rollback, and stage timing inspectionreasoning_trace_example.py: System 2 reasoning traces and storage patternsself_learning_ontology_example.py: Ontology promotion and governance workflowworking_holistic_example.py: Multi-memory demo across semantic, episodic, procedural, and working memory
Configuration
SmartMemory uses environment variables for configuration:
Environment Variables
Key environment variables:
OPENAI_API_KEY: OpenAI API key for embeddings and LLM extraction (auto-detected in lite mode)ANTHROPIC_API_KEY: Anthropic API key. When set, LLM extraction defaults toclaude-haiku-5-5; without it extraction usesopenai/gpt-oss-120bwhenGROQ_API_KEYis set, else the previous default model, and logs one warning. An explicit model setting still wins.ANTHROPIC_WORKSPACE_ID(optional) is sent as theanthropic-workspace-idheader for a key that is not workspace-scopedGROQ_API_KEY: Groq API key — alternative to OpenAI for LLM extraction (auto-detected in lite mode)
Server mode only:
FALKORDB_HOST: FalkorDB server host (default: localhost)FALKORDB_PORT: FalkorDB server port (default: 9010)REDIS_HOST: Redis server host (default: localhost)REDIS_PORT: Redis server port (default: 9012)
Dedicated vector domains (optional, default off): SmartMemory can route a domain's
embeddings into its own vector collection + embedder instead of the shared semantic_memory
collection. Each flag is read at registry-build time; flip it on only after the collection is
populated, so dev/CI never load the specialized model.
SMARTMEMORY_CODE_VECTOR_NAMESPACE: code →semantic_code(CodeRankEmbed, 768-dim).SMARTMEMORY_ENTERPRISE_VECTOR_NAMESPACE: enterprise docs →semantic_enterprise(bge-large, 1024-dim).
When a flag is on, an unscoped search() automatically fans out to that collection and
RRF-merges its hits with the base results (scoped/memory_type= queries are unaffected).
# Lite mode — only API key needed (optional, enables LLM extraction)
export OPENAI_API_KEY=your-api-key-here
# Server mode — also needs database hosts
export FALKORDB_HOST=localhost
export FALKORDB_PORT=9010
export REDIS_HOST=localhost
export REDIS_PORT=9012
Memory Evolution
SmartMemory includes built-in evolvers that automatically transform memories. In lite mode, evolution runs incrementally in the background — memories evolve as they're added, not just at the end of each pipeline run.
Available Evolvers
Core evolvers — memory type transitions and lifecycle:
- EpisodicToSemanticEvolver: Promotes stable facts to semantic memory
- EpisodicToZettelEvolver: Converts episodic events to Zettelkasten notes
- EpisodicDecayEvolver: Archives old episodic memories
- SemanticDecayEvolver: Prunes low-relevance semantic facts
- ZettelPruneEvolver: Merges duplicate or low-quality notes
- DecisionConfidenceEvolver: Decays confidence on stale decisions, auto-retracts below threshold
- OpinionSynthesisEvolver: Synthesizes opinions from accumulated observations
- ObservationSynthesisEvolver: Creates entity summaries from scattered facts
- OpinionReinforcementEvolver: Adjusts opinion confidence based on new evidence
- StaleMemoryEvolver: Flags memories as stale when referenced source code changes
Enhanced evolvers — neuroscience-inspired dynamics:
- ExponentialDecayEvolver: Time-based activation decay with configurable half-life
- RetrievalBasedStrengtheningEvolver: Memories accessed more frequently become harder to forget
- HebbianCoRetrievalEvolver: Reinforces edges between memories retrieved together ("neurons that fire together wire together")
- InterferenceBasedConsolidationEvolver: Similar competing memories interfere, strengthening the dominant one
- EnhancedWorkingToEpisodicEvolver: Context-aware pending→episodic transition with richer metadata
Sleep-cycle & agent evolvers — opt-in background re-derivation (REM/NREM analogs):
- MemoryConsolidationEvolver: Re-derives consolidated memories across an entity's scattered facts during an idle "sleep cycle"
- TemporalAgingEvolver: Rewrites future/planned facts into resolved past tense once their date passes ("going to Singapore in July" → "went to Singapore in July 2026") via reversible bi-temporal supersession — triggered by the passage of time, never re-parsing prose
- EvaluationEvolver: Writes evidence-derived per-
(agent, dimension, domain)performance scores via bi-temporal supersession; read back withget_evaluation()/list_evaluation_history() - SemanticToProceduralEvolver / ProceduralReinforcementEvolver / AnchorReconciliationEvolver: procedural promotion, usage-based strengthening, and spec-anchor drift reconciliation
Replaced by ConsolidationRouter (CORE-MEMORY-DYNAMICS-1 M1):
- The former
WorkingToEpisodicEvolverandWorkingToProceduralEvolverwere retired. Routing from thependingbucket toepisodic/proceduralnow happens at ingest time via theConsolidationRouterpipeline stage.
Evolvers run automatically as part of the memory lifecycle. See the examples directory for evolution demonstrations.
Plugin System
SmartMemory features a unified, extensible plugin architecture that allows you to customize and extend functionality. All plugins follow a consistent class-based pattern.
Built-in Plugins
SmartMemory auto-registers a small set of built-in plugins at startup, with additional specialist plugins available for opt-in use:
Auto-registered by default (loaded by PluginManager._load_builtin_plugins):
- Extractors:
LLMExtractor,LLMSingleExtractor,ConversationAwareLLMExtractor(always), plusSpacyExtractorandGLiNER2Extractorwhen their optional deps are importable - 6 Enrichers:
BasicEnricher,LinkExpansionEnricher,SentimentEnricher,TemporalEnricher,ExtractSkillsToolsEnricher,TopicEnricher - 9 Evolvers:
EpisodicToSemanticEvolver,EpisodicDecayEvolver,SemanticDecayEvolver,EpisodicToZettelEvolver,ZettelPruneEvolver,ExponentialDecayEvolver,InterferenceBasedConsolidationEvolver,RetrievalBasedStrengtheningEvolver,EvaluationEvolver - 1 Grounder:
WikipediaGrounder
Specialist plugins (selected by pipeline stage, opt-in feature, or the Studio evolver catalog — not auto-run in the idle cycle):
- Extractors:
GroqExtractor,DecisionExtractor,ReasoningExtractor - Enrichers:
UsageTrackingEnricher - Evolvers:
DecisionConfidenceEvolver,OpinionSynthesisEvolver,ObservationSynthesisEvolver,OpinionReinforcementEvolver,StaleMemoryEvolver,HebbianCoRetrievalEvolver,MemoryConsolidationEvolver,TemporalAgingEvolver,SemanticToProceduralEvolver,ProceduralReinforcementEvolver,AnchorReconciliationEvolver - Grounders:
PublicKnowledgeGrounder(Wikidata QIDs)
One lazy evolver catalog (
evolution/registry.py) serves Studio, explicit selections, and the idle cycle. Each spec declares whether it runs by default. The idle cycle keeps its existing default selection, while an explicit selection can reach every catalog key or registered spec path. Unregistered selections are refused with a warning, including built-in dotted paths and paths removed from the catalog. Plugin discovery registers approved external evolvers into the same catalog.
Export formats
from smartmemory import run_export resolves a registered format plugin before
calling a supplied selector. The selector supplies an authorized source, such as
a scoped ontology service or a bound memory.export_okf_bundle capability.
Built-in formats are okf, ontology_jsonld, ontology_ir_v1, ontology_legacy,
and admin_ndjson. Installed format plugins use the same operator allowlist as
other plugins through the smartmemory.plugins.formats entry point group.
run_export(name, select, audit=callback, **options) returns the format's document
or byte iterator. The audit callback receives completed, failed, or closed.
Stream completion is recorded only after exhaustion. Consumers must call
close() or aclose() when stopping early. OKF accepts bundle_dir and
archive_path paths and preserves the existing smartmemory-okf/ archive layout.
The adapter owns authorization and temporary-file cleanup. OKF still materializes
its selected items before streaming the archive.
Creating Custom Plugins
Create your own plugins by extending the base classes:
from smartmemory.plugins.base import EnricherPlugin, PluginMetadata
class MyCustomEnricher(EnricherPlugin):
@classmethod
def metadata(cls):
return PluginMetadata(
name="my_enricher",
version="1.0.0",
author="Your Name",
description="My custom enricher",
plugin_type="enricher",
dependencies=["some-lib>=1.0.0"],
security_profile="standard",
requires_network=False,
requires_llm=False
)
def enrich(self, item, node_ids=None):
# Your enrichment logic
item.metadata["custom_field"] = "value"
return item.metadata
See the examples directory for complete plugin examples.
Publishing Plugins
Publish your plugin as a Python package with entry points:
# pyproject.toml
[project.entry-points."smartmemory.plugins.enrichers"]
my_enricher = "my_package:MyCustomEnricher"
Install and use:
pip install my-smartmemory-plugin
# Then approve its exact entry point in config.json.
Installed third-party entry points require operator approval in config.json before
startup. Both the switch and an exact (group, name, distribution) entry are required:
{
"plugins": {
"allow_third_party": true,
"entry_point_allowlist": [
{"group": "smartmemory.plugins.enrichers", "name": "my_enricher", "distribution": "my-smartmemory-plugin"}
]
}
}
Distribution names use Python packaging normalization (case, hyphens, underscores).
Wildcards are unsupported. Refused entry points log a warning before any plugin code
is imported. Missing distribution identity is refused. Malformed or unreadable policy logs an error and refuses all third-party entries.
Malformed allowlist rows log an error and are skipped individually. Restart after changing policy. Built-in plugins load separately.
The default entry-point allowlist tracks core's pyproject.toml, whose plugin tables
currently contain no entries. This policy governs entry points, while explicitly
supplied plugin directories remain a separate trusted-code loading path.
Plugin Types
- ExtractorPlugin: Extract entities and relationships from text
- EnricherPlugin: Add metadata and context to memories
- GrounderPlugin: Link memories to external knowledge sources
- EvolverPlugin: Transform memories based on conditions
Approved entry-point plugins are discovered and registered at startup.
Plugin Security
SmartMemory includes a comprehensive security system for plugins:
- 4 Security Profiles:
trusted,standard(default),restricted,untrusted - Permission System: Control memory, network, file, and LLM access
- Resource Limits: Automatic timeout (30s), memory limits, network request limits
- Sandboxing: Isolated execution with security enforcement
- Static Validation: Detects security issues before execution
# Plugins are secure by default
PluginMetadata(
security_profile="standard", # Balanced security
requires_network=True, # Explicitly declare requirements
requires_llm=False
)
See the full documentation for complete security documentation.
Examples
See the examples/ directory for complete plugin examples:
custom_enricher_example.py- Sentiment analysis and keyword extractioncustom_evolver_example.py- Memory promotion and archivalcustom_extractor_example.py- Regex and domain-specific NERcustom_grounder_example.py- DBpedia and custom API grounding
Testing
Run the test suite:
# Run all tests
PYTHONPATH=. pytest -v tests/
# Run specific test categories
PYTHONPATH=. pytest tests/unit/
PYTHONPATH=. pytest tests/integration/
PYTHONPATH=. pytest tests/e2e/
# Run examples
PYTHONPATH=. python examples/memory_system_usage_example.py
PYTHONPATH=. python examples/conversational_assistant_example.py
API Reference
Progress Event Bus (PLAT-PROGRESS-1)
Unified progress emission via Redis Streams. Producers call emit(); consumers subscribe over SSE via /memory/progress/stream (see smart-memory-service). Stream key: sm:progress:<workspace_id>; Redis DB 1; MAXLEN default 10_000 (override via SMARTMEMORY_PROGRESS_MAXLEN).
from smartmemory.progress import bind, unbind, bound_progress, emit
# One emit anywhere in the process writes a ProgressEvent to the
# caller's workspace stream. run_id + scope come from contextvars set
# by bind() / bound_progress(); never passed as kwargs.
# Simple request-scoped pattern
bind(run_id="abc-123", scope="workspace:team-42")
try:
emit(kind="pipeline.stage", status="progress", stage="extract",
payload={"progress": 50, "message": "Extracting entities"})
finally:
unbind()
# Context-managed pattern — preferred for background tasks that run in
# a fresh asyncio context (e.g. FastAPI BackgroundTasks). bind()
# bindings in the scheduling handler don't propagate.
with bound_progress(run_id="abc-123", scope="workspace:team-42"):
emit(kind="pipeline.stage", status="ok", stage="extract",
payload={"progress": 100, "message": "Completed extraction"})
Status values: started | progress | ok | warn | error | skipped. Kinds are dotted producer-owned tags (pipeline.stage, evolver.result, studio.job, …); the transport has no registry.
SmartMemory Class
Main interface for memory operations:
class SmartMemory:
def __init__(
self,
scope_provider: Optional[ScopeProvider] = None,
vector_backend=None, # Any VectorStoreBackend; None uses default (FalkorDB)
cache=None, # Any cache-compatible object; e.g. NoOpCache()
observability: bool = True, # False disables Redis Streams emission and metrics
pipeline_profile=None, # PipelineConfig instance; PipelineConfig.lite() for zero-infra
entity_ruler_patterns=None, # Any object with get_patterns() -> dict[str, str]
)
# Primary API
def ingest(self, item, sync=True, **kwargs) -> str # Full pipeline
def add(self, item, **kwargs) -> str # Simple storage
def get(self, item_id: str) -> Optional[MemoryItem]
def search(
self,
query: str,
top_k: int = 5,
memory_type: str = None,
origin: str = None, # prefix filter, e.g. "user"
decompose_query: bool = False, # split compound queries
multi_hop: bool = False, # chained recursive retrieval
semantic_hops: bool = False, # LLM-driven hop planning
expertise: bool = False, # typed-dict expertise channel
include_superseded: bool = False,
as_of_date=None, # bi-temporal time travel (datetime | ISO-8601)
) -> List[MemoryItem]
def delete(self, item_id: str) -> bool
# Expertise-layer capture
def add_decision(self, content: str, **kwargs) -> str
def add_constraint(self, content: str, **kwargs) -> str
def add_learning(self, content: str, **kwargs) -> str
def add_opinion(self, content: str, subject: str, **kwargs) -> str
def add_observation(self, entity_id: str, fact_id: str, aspect: str = None) -> str
def add_reasoning(self, trace, source_item_id: str = None, **kwargs) -> str
# Graph Integrity (v0.3.8+)
def delete_run(self, run_id: str) -> int # Delete entities by pipeline run
def rename_entity_type(self, old: str, new: str) -> int # Ontology evolution
def merge_entity_types(self, sources: List[str], target: str) -> int
# Advanced
def run_clustering(self) -> dict
def run_evolution_cycle(self) -> None
def personalize(self, traits: dict = None, preferences: dict = None) -> None
def get_all_items_debug(self) -> Dict[str, Any]
# Lifecycle
def close(self) -> None # Clean shutdown (optional)
API Design:
ingest()- Full agentic pipeline: extract → store → link → enrich → evolve. Use for user-facing ingestion.add()- Simple storage: normalize → store → embed. Use for internal operations or when pipeline is not needed.
Scoping:
- OSS mode: No scoping needed, all data accessible
- For multi-tenant applications, pass a
ScopeProviderto enable automatic filtering - See the documentation for complete details
MemoryItem Class
Core data structure for memory storage:
@dataclass
class MemoryItem:
content: str
memory_type: str = 'semantic'
item_id: str = field(default_factory=lambda: str(uuid.uuid4()))
valid_start_time: Optional[datetime] = None
valid_end_time: Optional[datetime] = None
transaction_time: datetime = field(default_factory=datetime.now)
embedding: Optional[List[float]] = None
entities: Optional[list] = None
relations: Optional[list] = None
metadata: dict = field(default_factory=dict)
Security Metadata:
- For OSS usage, security metadata fields are not needed
- For multi-tenant applications, use a
ScopeProviderfor automatic metadata injection - See the documentation for details
Dependencies
Core Dependencies
SmartMemory requires the following key dependencies:
spacy: Natural language processing and deterministic entity extractionopenai: OpenAI SDK — the LLM client for extraction/classification chat completions (any OpenAI-compatible endpoint: OpenAI, Groq, etc.) and embeddings. Token usage is captured here for per-stage cost metering.litellm: legacy LLM client, superseded by the direct OpenAI SDK (kept for backward compatibility only)scikit-learn: Machine learning utilitiespydantic: Data validationpython-dateutil: Date/time handlingvaderSentiment: Sentiment analysisjinja2: Template rendering
Lite mode adds: usearch (vector search), uses Python's built-in sqlite3.
Server mode adds: falkordb (graph + vector storage), redis (caching).
Optional Dependencies
Install additional features as needed:
# Modes
pip install smartmemory-core[lite] # Zero-infra local mode (SQLite + usearch, no Docker)
pip install smartmemory-core[server] # Server mode (FalkorDB + Redis)
# Tools
pip install smartmemory-core[cli] # Command-line interface (add, search, rebuild)
pip install smartmemory-core[watch] # Vault watcher for auto-ingesting Markdown files
# Integrations
pip install smartmemory-core[slack] # Slack integration
pip install smartmemory-core[aws] # AWS integration
pip install smartmemory-core[wikipedia] # Wikipedia enrichment
# Everything
pip install smartmemory-core[all] # All optional features
Feedback
Bug reports and feature requests are welcome at support@smartmemory.ai.
📄 License
SmartMemory Lite is distributed as a compiled runtime under the SmartMemory Lite Runtime License (see LICENSE).
- Free for personal use, internal tools, evaluation, research, development, and non-commercial applications.
- Shipping the runtime inside a commercial product requires a Lite Redistribution Agreement, free during early access and for organizations below the revenue threshold in the agreement. Ask at help@smartmemory.ai.
- No source disclosure is required for your application.
The hosted service and platform are licensed separately.
Security
SmartMemory takes plugin security seriously. All plugins run in a sandboxed environment with:
- ✅ Permission checks - Plugins must declare what they access
- ✅ Resource limits - Automatic timeouts and memory limits
- ✅ Execution isolation - Sandboxed plugin execution
- ✅ Static analysis - Security validation before execution
External plugins use the standard security profile by default.
🔗 Links
- 📦 PyPI Package: https://pypi.org/project/smartmemory-core/
- 📚 Documentation: https://docs.smartmemory.ai
- 🐛 Support: support@smartmemory.ai
- 🔒 Security & Auth: See documentation
- 🔐 Plugin Security: See documentation
Get started with SmartMemory today!
pip install smartmemory
Explore the examples directory for complete demonstrations and use cases.
✅ Recently Completed
Incremental Evolution (v0.5.5)
- ✅ Event-driven evolution in lite mode: Memories evolve incrementally in the background as they're added, not just at pipeline end
- ✅ Backend API unification:
HebbianCoRetrievalEvolverrewritten from Cypher to backend API — works on both SQLite and FalkorDB - ✅
SmartMemory.close(): Optional clean shutdown for long-running scripts
Python effects
smartmemory.code.effects.scan_effects(directory, repo) produces deterministic Python boundary records, source hashes, call uncertainty and owning-code links without opening a memory store. Repository policy comes from .smartmemory-effects.json in the scanned checkout. link_to_entities maps owner references to existing code entities with matching source hashes. merge_effects reconciles measured runtime evidence.
Code Indexer (v0.5.x)
Code indexing reports clean, partial and failed file counts plus acceptance, staging and publication outcomes. Every hosted response derives counts from uploaded file evidence. Uncomputable evidence and missing server counts are displayed as unknown. Localized grammar partials with usable extraction can publish with visible spans and byte coverage. Hard collection or parse failures retain the prior index. Partial 1.x acceptance does not prove G16 complete-generation publication. Successful file extraction checkpoints are stored under SMARTMEMORY_CODE_CHECKPOINT_DIR (default ~/.cache/smartmemory/code-parse) and reused when source, configuration and parser versions match. Uploads derive file counts from diagnostic evidence, even when the summary is omitted. Unusable partials and malformed spans are refused before any write.
- ✅ AST-based Python parser: Extracts modules, classes, functions, FastAPI routes, pytest tests
- ✅ TypeScript/JavaScript parser: tree-sitter-based with React component/hook detection
- ✅ Cross-file call resolution: Symbol table resolves import aliases to entity
item_ids - ✅ Semantic code search: Vector embeddings on code entities for intent-based queries
- ✅ Git-anchored staleness:
commit_hashon entities + query-time drift detection - ✅ Memory↔code bridging:
REFERENCES_CODEedges link semantic memories to code entities - ✅ EntityRuler seeding: Code class/function names auto-added to pattern dictionary
Query Decomposition (v0.5.x)
- ✅ Compound query splitting: "auth flow and caching strategy" → 2 sub-queries
- ✅ Cross-query RRF merge: Independent search per sub-query, reciprocal rank fusion
- ✅
decompose_query=TrueonSmartMemory.search(), REST, and MCP surfaces
Zero-Infra Lite Mode (v0.4.x)
- ✅
smartmemory-core[lite]: SQLite + usearch backend — no Docker, no FalkorDB, no Redis required - ✅
create_lite_memory(): Factory function fromsmartmemory.tools.factoryfor zero-config setup - ✅
PipelineConfig.lite(llm_enabled=None): LLM extraction auto-detected fromOPENAI_API_KEY/GROQ_API_KEY - ✅ Graceful degradation: VersionTracker, TemporalQueries, Grounding all work on SQLite backend
- ✅ Constructor injection:
vector_backend,cache,observability,pipeline_profile,entity_ruler_patterns
Unified Pipeline v2 (v0.3.x)
- ✅ 11-stage composable pipeline: classify → coreference → simplify → entity_ruler → llm_extract → ontology_constrain → store → link → enrich → ground → evolve
- ✅ Self-learning ontology: OntologyGraph with three-tier status (seed → provisional → confirmed), six-gate promotion, hot-reloadable patterns
- ✅ Breakpoint execution:
run_to(),run_from(),undo_to()for debugging and resumption - ✅ Async mode:
ingest(sync=False)with Redis Streams transport
Memory Types & Reasoning (v0.2.x–v0.3.x)
- ✅ Decision memory: Confidence tracking, provenance chains, causal chain traversal
- ✅ Opinion/Observation synthesis: Beliefs with confidence scores, entity summaries from scattered facts
- ✅ Reasoning traces: Chain-of-thought capture with auto-detection via classify stage
- ✅ Graph validation: Schema enforcement, health metrics, inference engine, symbolic reasoning
See CHANGELOG.md for complete version history.
Check the documentation for the latest updates.
Lexical search migration
Search uses lexical with default weight 0.8. Replace removed contains and keyword-bm25 channel weights explicitly. A zero weight disables lexical, and omission preserves the existing default/profile behavior. Required lexical unavailability fails the whole search. Service callers receive 400 for query/name validation and 503 for unavailable indexes.
Lite is single-tenant. Graph-first retrieval conditionally fills from FTS5 lexical and vector search. Bounded refill can underfill, and regex-text still scans. CORE-FACT-INDEX-1 separately owns deferred fact ranking and path expansion.
Quiesce old writers before first-open indexing. Verify the engine capability pin and use sm rebuild --lexical for recovery. Migration, targets and measured limitations.
Reader recall presets (SM-RECALL-PROFILES-1)
Use recall_pack(preset="brief") in core or memory_recall_pack(preset="brief")
over MCP. A reader preset selects ordered sections and a default token budget.
Pass budget_tokens to override the default.
| Preset | Default tokens | Sections in priority order |
|---|---|---|
brief |
2000 | constraints, active_plan, anchors, learned, tier1_user |
resume |
3000 | active_plan, anchors, last_session, snapshot, constraints, learned |
review |
4000 | constraints, learned, tier1_user, notes |
pane |
600 | constraints, active_plan, anchors, last_session |
preset and explicit sections are mutually exclusive. Calls without a reader
preset, including wakeup, still require an explicit budget. The manifest names
the preset, resolved budget and section caps. Budget loss and sections unavailable
for as_of reads retain their WARNING logs and manifest accounting. Search agent
recall profiles continue to control memory-type weights independently.
MCP expands the four reader presets into the existing sections and cap_tokens
request fields for local, remote and hosted calls. Remote and hosted use requires
the SM-LEARNED-TOOLS-1 service dependency for learned and constraint sections.
Local detection rules
SM-POLICY-COMPILER-1 extends the existing adherence Detection patterns catalog.
Use smartmemory.adherence.compile_rule_patterns(directory) for validated rules
and named unusable records. load_rule_patterns keeps its existing list shape.
Regexes use Python re syntax and inline flags. Phrases match case-insensitively.
exclude_regex whitelists matching scan text. Suppression checks the configured
recent user turn window. Each rule runs in an isolated worker with a 250 ms
wall-clock deadline, including positive, exclusion and suppression regexes.
Timeouts log WARNING with the rule name, mark coverage incomplete and retain
results from successful sibling rules.
Source files and compiled records are limited to 64 KiB each. The source and compiled catalogs each have a 1 MiB budget, with at most 128 candidate rule files and 4,096 scanned directory entries. Each predicate list has at most 128 entries, each at most 4,096 characters. Checks accept at most 64 Ki characters per draft or user turn, 32 user turns and 256 Ki characters of combined text. JSON stdin is limited to 1 Mi characters. Limit losses are explicit warnings and exit 2 for draft checks. Rejected rules do not discard successfully compiled siblings.
Set SMARTMEMORY_RULES_DIR and run python -I -m smartmemory.adherence.session to
print a session check card. Add --check and provide JSON on stdin with
response and optional recent_user_turns, ordered oldest to newest, to check
a draft. Exit 0 means no compiled rule matched with full rule coverage, exit 1
means a violation, and exit 2 means incomplete coverage or a configuration error.
Malformed individual entries log WARNING and remaining valid entries still run.
The checker never logs candidate text or uses a store or model.
The orient integration primes advisory response checks. SessionStart has no future response to inspect, so this is not an automatic blocking gate. Each session must choose its own directory. Filesystem access stays local. Directory components are pinned with descriptors and symlinks are refused in the selected path and rule entries. Enumeration and reads use the pinned directory, so a concurrent parent replacement cannot redirect reads. No hosted filesystem tool or service route is added.
Call and write tracking
Provider adapters use the event gateway in smartmemory.observability.tracking. One terminal ai.call event describes each leaf SDK or local inference dispatch, including failures. Native SDK retries remain within that dispatch until BILL-6 owns HTTP attempt metering. Prompts, responses and exception bodies are excluded.
Mutation funnels emit write.operation events. Background execution emits job.run events when it starts and finishes. Events use the active in-process sink or the bound workspace progress stream. Missing delivery receipts produce an untracked WARNING naming the source call site. Custom sinks should return True from emit after accepting an event. Queued delivery can still be refused later, in which case the built-in queue logs the call site.
Tracking failures preserve the application result or exception. If a warning handler fails, tracking attempts one stderr message naming the lost event. SQLite statement tracking runs at the cursor, with no events or transport calls for proven reads and transaction controls.
The census tests compare provider, write and job sites against reviewed manifests and reject new unguarded sites. A legacy trace_span alone does not prove delivery. Static discovery resolves known SDK calls, HTTP client aliases and literal SQL mutations. Arbitrary dynamic dispatch and new inference libraries still require review. Meter wiring remains deferred, with a strict expected-failure check against the real SQLite ledger.
Conflict pair review (core API, post-cut)
AssertionChallenger persists deferred challenges and manual-resolution fallbacks
in the bound SQLite or FalkorDB database. List and settle pairs in that same scope:
from smartmemory.reasoning.challenger import AssertionChallenger
challenger = AssertionChallenger(memory)
for review in challenger.list_reviews(limit=50, offset=0):
challenger.settle_review(
review["id"], action="keep_both",
reason="Both prices apply to different plans", actor="human:reviewer",
)
Actions are keep_a (existing fact), keep_b (incoming assertion), and
keep_both. A nonblank reason and actor are required. Resolved receipts are
available with list_reviews(status="resolved"). A settled decision cannot be
overwritten. This core surface records adjudication of the pair. It does not
admit the incoming assertion as a memory or retire an existing memory. New HTTP,
SDK, MCP and supersession-proposal consumers remain separate work.
Personal sources are refused before queue publication. Reads and settlement also
recheck source admission, so legacy personal receipts remain hidden. A page can
contain fewer than limit rows after source admission.
AssertionChallenger(..., review_enqueuer=adapter) accepts the public
ConflictReviewEnqueuer protocol from smartmemory.reasoning. Its only operation
is enqueue(conflict, *, trigger), returning a durable receipt or raising.
Secure producers supply an authenticated common adapter that re-reads and admits
the source before trusted persistence. Direct core callers retain the default
SQLite or FalkorDB queue. Hosted list and settlement need a separate authority
contract. The adapter does not expose a graph, backend or connection.
Human procedures
SM-PROCEDURE-TYPE-1 adds structured runbooks through a narrow scoped store. Supply an already authorized memory instance:
from smartmemory.procedures.human import ProcedureManager
from smartmemory.procedures.store import MemoryProcedureStore
procedures = ProcedureManager(MemoryProcedureStore(memory))
procedure = procedures.create(
name="Restore database",
steps=["Check backup", "Restore database", "Check health"],
preconditions=["A verified backup exists"],
expected_outcome="Database passes health checks",
)
matches = procedures.list(task="restore database")
result = procedures.record_outcome(procedure["procedure_id"], success=True, note="Health checks passed")
Outcomes remain separate durable episodic records. Confidence rises after success and falls after failure. Responses include the latest 20 outcomes. Candidates require explicit promotion, and dismissal is terminal. This surface stores human runbooks and does not execute their steps. The paired MCP change is SM-PROCEDURE-MCP-1.
Metadata
Release files for smartmemory-core 2.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Actions Reset | |||
|---|---|---|---|---|
| smartmemory_core-2.0.1-cp314-none-any.whl | CPython 3.14 | none | any | Details for smartmemory_core-2.0.1-cp314-none-any.whl |
| smartmemory_core-2.0.1-cp313-none-any.whl | CPython 3.13 | none | any | Details for smartmemory_core-2.0.1-cp313-none-any.whl |
| smartmemory_core-2.0.1-cp312-none-any.whl | CPython 3.12 | none | any | Details for smartmemory_core-2.0.1-cp312-none-any.whl |
| smartmemory_core-2.0.1-cp311-none-any.whl | CPython 3.11 | none | any | Details for smartmemory_core-2.0.1-cp311-none-any.whl |
Total release size: 32.3 MB
Release files / smartmemory_core-2.0.1-cp314-none-any.whl
| Download URL | smartmemory_core-2.0.1-cp314-none-any.whl |
|---|---|
| Size | 8.5 MB |
| Tags | CPython 3.14 |
|
SHA-256 checksum How to use checksums |
7558cb2e55441aea0820f440dfaddff2820237fcf329cd0002d085cbc81a169a
|
|
BLAKE2b-256 checksum How to use checksums |
cbc3a59d4f034bee1a8c75a345149b43ad2cbe004f0327e4c1ef7cc47409506b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.7
|
Release files / smartmemory_core-2.0.1-cp313-none-any.whl
| Download URL | smartmemory_core-2.0.1-cp313-none-any.whl |
|---|---|
| Size | 7.8 MB |
| Tags | CPython 3.13 |
|
SHA-256 checksum How to use checksums |
18e0312298651579f498958c4992f7fc305ea6a480fae89a3c3a3c3359d326c9
|
|
BLAKE2b-256 checksum How to use checksums |
c919c8d5e8988c5a8ad710346a06a7da14e34b1fbd6ffc025988d25517238ba2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.7
|
Release files / smartmemory_core-2.0.1-cp312-none-any.whl
| Download URL | smartmemory_core-2.0.1-cp312-none-any.whl |
|---|---|
| Size | 7.8 MB |
| Tags | CPython 3.12 |
|
SHA-256 checksum How to use checksums |
bb61a0c3ed0b668d9f69df49acc1844656189ae48b0bce2d0419e65148c39e61
|
|
BLAKE2b-256 checksum How to use checksums |
fc007ca3b2592dc4d48d5993431e21bb64ebd3d6cd431e436fc23ed84fdcafe7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.7
|
Release files / smartmemory_core-2.0.1-cp311-none-any.whl
| Download URL | smartmemory_core-2.0.1-cp311-none-any.whl |
|---|---|
| Size | 8.2 MB |
| Tags | CPython 3.11 |
|
SHA-256 checksum How to use checksums |
e083e73886569cc20795f96336e0d94212ddd26ed50e617c91695d719769b4b9
|
|
BLAKE2b-256 checksum How to use checksums |
d346ce2f302f3fe788bd207f829cb035d4652b2f368888ddbee62544144dad5c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.7
|
Security for smartmemory-core 2.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.