LSS — Local Semantic Search

Hybrid search over local files. BM25 + embeddings + Reciprocal Rank Fusion. Real-time file watching. Runs offline or with OpenAI.

lss "authentication JWT"              # search current directory
lss "deploy kubernetes" ~/Projects    # search a specific path
lss "rate limiting" --json            # machine-readable output

0.91 NDCG@10 on our golden set. Beats ColBERTv2, Voyage-2, and Cohere embed-v3 on BEIR SciFact. See EVALS.md.

Install

# One-liner (auto-detects pipx/uv/pip)
curl -fsSL https://raw.githubusercontent.com/kortix-ai/lss/main/install.sh | bash

Or directly:

pipx install local-semantic-search       # recommended
pip install local-semantic-search
uv tool install local-semantic-search

Embedding provider

Default: OpenAI — if OPENAI_API_KEY is set, lss uses it automatically.

export OPENAI_API_KEY="sk-..."   # add to ~/.zshrc or ~/.bashrc

Offline alternative:

pip install 'local-semantic-search[local]'
lss config provider local

Uses bge-small-en-v1.5 (384d, ~125 MB). No API key, no network, no cost. Within 0.3% of OpenAI on quality, 8x faster.

Usage

Search

lss "query"                              # current directory
lss "query" ~/Documents                  # explicit path
lss "auth JWT" "deploy k8s"              # multiple queries
lss "config" --json                      # JSON output
lss "error" -k 5                         # top 5

# Filters (applied without re-indexing)
lss "auth" -e .py -e .ts                 # only these extensions
lss "config" -E .json -E .yaml           # exclude extensions
lss "user data" -x '\d{4}-\d{2}-\d{2}'  # exclude chunks matching regex
lss "auth" -e .py -x "test_"            # combine filters

First search auto-indexes. Subsequent searches use cached embeddings.

Index & manage

lss index ~/Projects                     # index without searching
lss status                               # DB stats, provider, config
lss ls                                   # list indexed files
lss sweep --clear-all                    # wipe database

lss watch add ~/Documents                # for lss-sync daemon
lss exclude add "*.log"                  # glob exclusion
lss include add .rst                     # custom extension
lss config provider local                # switch provider
lss eval                                 # run quality benchmarks
lss update                               # check for updates

File watcher

lss-sync                                 # watch configured paths
lss-sync --watch ~/Projects              # watch specific path

How it works

query → BM25 (FTS5 + custom rescore) ─┐
      → Embedding (OpenAI or local)  ──┤→ RRF → boosts → MMR → results

1. BM25 — FTS5 retrieves candidates, custom re-scorer ranks with TF saturation + IDF (k1=1.2, b=0.75)
2. Embedding — OpenAI text-embedding-3-small (256d) or local bge-small-en-v1.5 (384d), cached permanently
3. RRF — Reciprocal Rank Fusion merges both ranked lists
4. Boosts — Jaccard overlap, phrase matching, digit co-mention
5. MMR — Maximal Marginal Relevance for diversity

See ARCHITECTURE.md for full pipeline detail.

Supported formats

Extraction via pdfminer.six, python-docx, openpyxl, python-pptx, beautifulsoup4, and built-in SQLite row extraction. Unknown extensions skipped by default; add with lss include add .ext.

Search quality

BEIR SciFact (5,183 docs, 300 queries):

Full results: EVALS.md

Performance

Configuration

Config file: ~/.lss/config.json

Programmatic use

from semantic_search import semantic_search
from lss_store import ingest_many, discover_files

all_files, new_files, _ = discover_files("/path/to/project")
ingest_many(new_files)
results = semantic_search("/path/to/project", ["JWT authentication"])

Tests

361+ tests covering extraction, filtering, chunking, CLI, e2e, file watching, providers, and search quality.

python -m pytest tests/ -x -q

License

Apache-2.0