okb 1.0.0

Personal knowledge base with semantic search for LLMs

Owned Knowledge Base (OKB)

A local-first semantic search system for personal documents with Claude Code integration via MCP.

Installation

pip install okb

Or from source:

git clone https://github.com/yourusername/okb
cd okb
pip install -e .

Quick Start

# 1. Start the database
okb db start

# 2. (Optional) Deploy Modal embedder for faster batch ingestion
okb modal deploy

# 3. Ingest your documents
okb ingest ~/notes ~/docs

# 4. Configure Claude Code MCP (see below)

CLI Commands

Command	Description
okb db start	Start pgvector database container
okb db stop	Stop database container
okb db status	Show database status
okb db destroy	Remove container and volume (destructive)
okb ingest <paths>	Ingest documents into knowledge base
okb ingest <paths> --local	Ingest using CPU embedding (no Modal)
okb serve	Start MCP server (stdio, for Claude Code)
okb serve --http	Start HTTP MCP server with token auth
okb watch <paths>	Watch directories for changes
okb config init	Create default config file
okb config show	Show current configuration
okb modal deploy	Deploy GPU embedder to Modal
okb token create	Create API token for HTTP server
okb token list	List tokens for a database
okb token revoke	Revoke an API token
okb sync list	List available API sources (plugins)
okb sync run <sources>	Sync data from external APIs
okb sync status	Show last sync times
okb rescan	Check indexed files for changes, re-ingest stale
okb rescan --dry-run	Show what would change without executing
okb rescan --delete	Also remove documents for missing files
okb llm status	Show LLM config and connectivity
okb llm deploy	Deploy Modal LLM for open model inference
okb llm clear-cache	Clear LLM response cache

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         INGESTION (Burst GPU)                       │
│                                                                     │
│  Local Files → Contextual Chunking → Modal (GPU T4) → pgvector     │
│                                                                     │
│  ~/notes/project-x/api-design.md                                   │
│       ↓                                                             │
│  "Document: API Design Notes                                        │
│   Project: project-x                                                │
│   Section: Authentication                                           │
│   Content: Use JWT tokens with..."                                  │
│       ↓                                                             │
│  [0.23, -0.41, 0.87, ...]  → pgvector                              │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│                      RETRIEVAL (Always-on, Local)                   │
│                                                                     │
│  Claude Code → MCP Server → CPU Embedding → pgvector → Results     │
│                                                                     │
│  "How do I handle auth?"                                            │
│       ↓                                                             │
│  [0.19, -0.38, 0.91, ...]  (local CPU, ~300ms)                     │
│       ↓                                                             │
│  Cosine similarity search → Top 5 chunks with context              │
└─────────────────────────────────────────────────────────────────────┘

Configuration

Configuration is loaded from ~/.config/okb/config.yaml (or $XDG_CONFIG_HOME/okb/config.yaml).

Create default config:

okb config init

Example config:

databases:
  personal:
    url: postgresql://knowledge:localdev@localhost:5433/personal_kb
    default: true    # Used when --db not specified (only one can be default)
    managed: true    # okb manages via Docker
  work:
    url: postgresql://knowledge:localdev@localhost:5433/work_kb
    managed: true

docker:
  port: 5433
  container_name: okb-pgvector

chunking:
  chunk_size: 512
  chunk_overlap: 64

Use --db <name> to target a specific database with any command.

Environment variables override config file settings:

• KB_DATABASE_URL - Database connection string
• OKB_DOCKER_PORT - Docker port mapping
• OKB_CONTAINER_NAME - Docker container name

Project-Local Config

Override global config per-project with .okbconf.yaml (searched from CWD upward):

# .okbconf.yaml
default_database: work  # Use 'work' db in this project

extensions:
  skip_directories:     # Extends global list
    - test_fixtures

Merge: scalars replace, lists extend, dicts deep-merge.

LLM Integration (Optional)

Enable LLM-based document classification and filtering:

llm:
  provider: claude          # "claude", "modal", or null (disabled)
  model: claude-haiku-4-5-20251001
  timeout: 30
  cache_responses: true

Providers:

Provider	Setup	Cost
claude	export ANTHROPIC_API_KEY=...	~$0.25/1M tokens
modal	okb llm deploy	~$0.02/min GPU

For Modal (no API key needed):

llm:
  provider: modal
  model: meta-llama/Llama-3.2-3B-Instruct

Pre-ingest filtering - skip low-value content during sync:

plugins:
  sources:
    dropbox-paper:
      llm_filter:
        enabled: true
        prompt: "Skip meeting notes and drafts"
        action_on_skip: discard  # or "archive"

CLI commands:

okb llm status              # Show config and connectivity
okb llm deploy              # Deploy Modal LLM (for provider: modal)
okb llm clear-cache         # Clear response cache

Claude Code MCP Config

stdio mode (default)

Add to your Claude Code MCP configuration:

{
  "mcpServers": {
    "knowledge-base": {
      "command": "okb",
      "args": ["serve"]
    }
  }
}

HTTP mode (for remote/shared servers)

First, start the HTTP server and create a token:

# Create a token
okb token create --db default -d "Claude Code"
# Output: okb_default_rw_a1b2c3d4e5f6g7h8

# Start HTTP server
okb serve --http --host 0.0.0.0 --port 8080

Then configure Claude Code to connect via SSE:

{
  "mcpServers": {
    "knowledge-base": {
      "type": "sse",
      "url": "http://localhost:8080/sse",
      "headers": {
        "Authorization": "Bearer okb_default_rw_a1b2c3d4e5f6g7h8"
      }
    }
  }
}

MCP Tools (Available in Claude Code)

Tool	Purpose
search_knowledge	Semantic search with natural language queries
keyword_search	Exact keyword/symbol matching
hybrid_search	Combined semantic + keyword (RRF fusion)
get_document	Retrieve full document by path
list_sources	Show indexed document stats
list_projects	List known projects
recent_documents	Show recently indexed files
save_knowledge	Save knowledge from Claude for future reference
delete_knowledge	Delete a Claude-saved knowledge entry
get_actionable_items	Query tasks/events with structured filters

Contextual Chunking

Documents are chunked with context for better retrieval:

Document: Django Performance Notes
Project: student-app          ← inferred from path or frontmatter
Section: Query Optimization   ← extracted from markdown headers
Topics: django, performance   ← from frontmatter tags
Content: Use `select_related()` to avoid N+1 queries...

Frontmatter Example

---
tags: [django, postgresql, performance]
project: student-app
category: backend
---

# Query Optimization

Use `select_related()` for foreign keys...

Cost Estimate

Component	Local	Cloud Alternative
pgvector	$0	~$15-30/mo (CloudSQL)
MCP Server	$0	~$5/mo (small VM)
Modal embedding	~$0.50-2/mo	N/A
Total	~$1-2/mo	~$20-35/mo

Development

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Lint and format
ruff check . && ruff format .

Plugin System

OKB supports plugins for custom file parsers and API data sources (GitHub, Todoist, etc).

Creating a Plugin

# File parser plugin
from okb.plugins import FileParser, Document

class EpubParser:
    extensions = ['.epub']
    source_type = 'epub'

    def can_parse(self, path): return path.suffix.lower() == '.epub'
    def parse(self, path, extra_metadata=None) -> Document: ...

# API source plugin
from okb.plugins import APISource, SyncState, Document

class GitHubSource:
    name = 'github'
    source_type = 'github-issue'

    def configure(self, config): ...
    def fetch(self, state: SyncState | None) -> tuple[list[Document], SyncState]: ...

Registering Plugins

In your plugin's pyproject.toml:

[project.entry-points."okb.parsers"]
epub = "okb_epub:EpubParser"

[project.entry-points."okb.sources"]
github = "okb_github:GitHubSource"

Configuring API Sources

# ~/.config/okb/config.yaml
plugins:
  sources:
    github:
      enabled: true
      token: ${GITHUB_TOKEN}  # Resolved from environment
      repos: [owner/repo1, owner/repo2]
    dropbox-paper:
      enabled: true
      token: ${DROPBOX_TOKEN}
      folders: [/]            # Optional: filter to specific folders

License

MIT