openlmlib 0.2.0

Local cross-project knowledge and research library for LLM workflows

OpenLMlib

Local knowledge and research library for LLM workflows

Store, retrieve, and collaborate on findings with semantic search, full-text search, and multi-agent collaboration sessions.

📚 Full Documentation · Quickstart · MCP Tools · CollabSessions

---

Features

• Knowledge Base: SQLite metadata + JSON findings + FAISS/Numpy vector index
• Semantic Retrieval: Multi-phase retrieval with semantic + lexical search, deduplication, and reranking
• MCP Server: 42 tools for AI assistants (11 core + 31 collaboration)
• CollabSessions: Multi-agent collaboration with message passing, artifacts, and templates
• CLI: Full command-line interface for management and diagnostics
• Portable: Findings exportable as JSON, easy backup and restore

---

Quickstart

Installation

Option 1: npm (Recommended)

npm install -g openlmlib
openlmlib setup  # Initialize and configure

Option 2: pipx

pipx install openlmlib
openlmlib setup

Option 3: From Source

git clone https://github.com/Vedant9500/LMlib.git
cd LMlib
pip install -e .
openlmlib setup

Note: The embedding model (~100-500MB) downloads during setup, not installation.

First Steps

# Check health
openlmlib doctor

# Add a finding
openlmlib add \
  --project myproj \
  --claim "Contextual chunking improves retrieval by 15-30%" \
  --confidence 0.85 \
  --evidence "https://arxiv.org/example" \
  --reasoning "Benchmarks show context-aware chunking outperforms fixed-size"

# Search
openlmlib query "retrieval techniques" --final-k 5

# List findings
openlmlib list --limit 20

Configure AI Assistants

# Interactive setup (recommended)
openlmlib setup

# Or configure specific IDEs
openlmlib mcp-config --ide vscode --ide cursor

Supported clients: VS Code, Cursor, Claude Desktop, Kiro, Windsurf, Zed, Cline, and more.

---

What Can You Do?

📚 Build a Knowledge Base

Store findings from research, experiments, or analysis with structured metadata:

openlmlib add \
  --project retrieval \
  --claim "Dynamic chunk sizing reduces hallucination by 20%" \
  --confidence 0.78 \
  --evidence "https://example.com/study" \
  --reasoning "Adaptive chunk size based on query complexity..." \
  --caveats "Requires query complexity estimation" \
  --tags retrieval,chunking,evaluation

🔍 Retrieve with Context

Multi-phase retrieval combines semantic similarity, lexical matching, and recency:

# Semantic search with reasoning traces
openlmlib query "contextual retrieval" \
  --final-k 5 \
  --reasoning-trace

# With filters
openlmlib query "retrieval" \
  --project myproj \
  --tags retrieval \
  --confidence-min 0.8

🤖 Use with AI Assistants

42 MCP tools let AI assistants securely access and modify your knowledge base:

Core Tools (11):

• openlmlib_init, openlmlib_health - Setup and diagnostics
• openlmlib_add_finding, openlmlib_delete_finding - Write operations (require confirmation)
• openlmlib_retrieve, openlmlib_search_fts - Retrieval and search
• openlmlib_list_findings, openlmlib_get_finding - Browse findings
• openlmlib_retrieve_context - Format findings for LLM prompts
• openlmlib_evaluate_dataset, openlmlib_help - Utilities

📖 See all 42 tools →

👥 Multi-Agent Collaboration

CollabSessions enable structured collaboration between multiple LLM agents:

# Create session from template
openlmlib-mcp --call collab_create_session_from_template '{
  "template_id": "deep_research",
  "title": "Research on Retrieval",
  "created_by": "gpt-4"
}'

# Join session
openlmlib-mcp --call collab_join_session '{
  "session_id": "sess_20260409_abc12345",
  "model": "claude-3",
  "role": "worker"
}'

# Send and receive messages
openlmlib-mcp --call collab_send_message '{...}'
openlmlib-mcp --call collab_poll_messages '{...}'

# Add artifacts (reports, analysis)
openlmlib-mcp --call collab_add_artifact '{...}'

Available Templates:

• deep_research - Comprehensive research (5 steps, 5 agents)
• code_review - Multi-agent code review (5 steps, 4 agents)
• market_analysis - Market/competitor analysis (4 steps, 4 agents)
• incident_investigation - Root cause analysis (4 steps, 3 agents)
• literature_review - Academic literature review (6 steps, 5 agents)

📖 Full CollabSessions guide →

---

Architecture

OpenLMlib
├── Knowledge Base
│   ├── SQLite (metadata, full-text search)
│   ├── FAISS/Numpy (vector index)
│   └── JSON findings (portable, human-readable)
│
├── MCP Server (42 tools)
│   ├── 11 core library tools
│   └── 31 collaboration tools
│
├── CLI
│   ├── Setup and configuration
│   ├── Finding management
│   └── Diagnostics (doctor command)
│
└── CollabSessions
    ├── Message bus (SQLite + JSONL)
    ├── Artifact store
    ├── Session templates
    └── Context compaction

---

Documentation

📚 Complete documentation is in the docs/ folder:

• docs/README.md - Documentation index and quick reference
• docs/MCP_TOOLS.md - Complete reference for all 42 MCP tools
• docs/COLLAB_SESSIONS.md - Multi-agent collaboration guide
• docs/SYSTEM_PROMPT.md - Agent instruction templates

---

CLI Reference

# Setup and diagnostics
openlmlib setup          # First-run bootstrap
openlmlib doctor         # Health check
openlmlib --version      # Show version

# Knowledge base
openlmlib init           # Initialize storage
openlmlib add            # Add finding
openlmlib list           # List findings
openlmlib get            # Get finding details
openlmlib query          # Semantic retrieval
openlmlib delete         # Delete finding

# Collaboration
openlmlib-mcp            # MCP server (auto-configured)

# Backup and restore
openlmlib backup         # Create backup
openlmlib restore        # Restore from backup

---

Configuration

Global Install

• Settings: ~/.openlmlib/config/settings.json
• Data: ~/.openlmlib/data/

Local/Dev Install

• Pass --settings /path/to/settings.json

---

Uninstallation

# Remove package
npm uninstall -g openlmlib    # if installed via npm
pipx uninstall openlmlib      # if installed via pipx
pip uninstall openlmlib       # if installed from source

# Remove data (optional)
rm -rf ~/.openlmlib           # global install data
rm -rf data/                  # local install data

---

Development

git clone https://github.com/Vedant9500/LMlib.git
cd LMlib
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -e ".[dev,faiss]"

# Run tests
python -m pytest tests/ -v

# Run MCP server manually
python -m openlmlib.mcp_server --settings ./config/settings.json

---

Notes

• Vector Search: Uses FAISS if installed, otherwise Numpy fallback
• Embedding Model: sentence-transformers/all-MiniLM-L6-v2 (default)
• Python: Requires 3.10+
• Global vs Local: Global installs use ~/.openlmlib/, local uses project data/

---

Releases

• Versioning: Semantic versioning (MAJOR.MINOR.PATCH)
• Changelog: CHANGELOG.md
• Release process: RELEASE.md

---

Contributing

See CONTRIBUTING.md for development workflow and guidelines.

---

License

MIT License - see LICENSE