lexical-workspace 0.1.1

Local AI workspace: LLM chat with RAG, persistent memory, image & audio generation, agents, and an interactive CLI.

Lexical LLM Assistant

A fast, CPU-only local LLM assistant that learns from your Markdown files and conversations. Everything runs on your own machine — no GPU, no API keys, no network calls by default.

• CPU-Only Inference — llama-cpp-python with GGUF models (no GPU required)
• Markdown Learning — ingest .md files with header-aware chunking + hybrid search
• Persistent Memory — remember facts across sessions (/memory, remember, auto-capture)
• Hybrid Retrieval — BM25 (sparse) + FAISS dense cosine + cross-encoder rerank
• Local Image Generation — Stable Diffusion (txt2img / img2img), interactive settings
• Local Audio Generation — MusicGen / ACE-Step text-to-music, interactive settings
• LoRA Training — optional fine-tuning on your own data
• Agent Mode — local tool-use (calculator, file read, doc search, open app, shell)

---

Table of Contents

1. Quick Start
2. Running the CLI
3. Command Reference
   • System
   • Chat & Memory
   • Documents
   • Models
   • LoRA
   • Images
   • Audio
   • Agent
   • Config & Stats
4. Natural-Language Memory
5. Configuration
6. CPU Optimization
7. LoRA Fine-Tuning
8. Project Structure
9. Running Tests
10. Troubleshooting
11. License & Acknowledgments

---

Quick Start

1. Install Dependencies

cd lexical-llm
pip install -r requirements.txt
# Optional: install as a package so `lexical` / `lexical-train` / ... entry
# points are available.
pip install -e .
# Optional: enable the ACE-Step audio backend (slow on CPU; needs newer deps)
pip install ace-step

Dependency note: torch==2.2.2 (CPU), transformers==4.41.2, and diffusers==0.30.3 are pinned on purpose. Do not bump diffusers — the audio and image backends are validated against these versions. ACE-Step is kept as an opt-in extra so its (potentially newer) transformers requirement never affects MusicGen or image generation unless you install it and select it.

2. Get a Model

Download a small GGUF model (1B–3B params, Q4_K_M quantization recommended) into models/:

# Llama 3.2 3B Instruct (~2GB)
wget -P models \
  https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q4_K_M.gguf

# Phi-3 Mini 3.8B (~2.3GB)
wget -P models \
  https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf

# Gemma 2 2B (~1.5GB)
wget -P models \
  https://huggingface.co/bartowski/gemma-2-2b-it-GGUF/resolve/main/gemma-2-2b-it-Q4_K_M.gguf

You can also let the CLI download one for you later with /download-model <url or hf-repo:file>.

3. (Optional) First-Time Setup

python -m lexical_llm.cli --ingest     # alias: lexical-setup (creates dirs + config)

/setup inside the CLI launches an interactive wizard (model pick + download + config.yaml). Non-interactive contexts fall back to silent setup.

4. Add Documents

mkdir -p data/docs
cp your-notes/*.md data/docs/

5. Ingest

python -m lexical_llm.scripts.ingest        # or: lexical-ingest

…or run /ingest from inside the CLI (rebuilds the hybrid index automatically).

6. Run

python -m lexical_llm.cli

After pip install -e . you can also just run lexical.

---

Running the CLI

Action	Command
Launch chat	python -m lexical_llm.cli (or lexical)
Custom config	python -m lexical_llm.cli --config path/to/config.yaml
Override model	python -m lexical_llm.cli --model models/foo.gguf
Set thread count	python -m lexical_llm.cli --threads 6
Ingest then exit	python -m lexical_llm.cli --ingest
Standalone ingest	python -m lexical_llm.scripts.ingest (or lexical-ingest)
Standalone setup	python -m lexical_llm.cli --ingest (or lexical-setup)
LoRA training	python -m lexical_llm.train_lora ... (or lexical-train)

Relative paths (config, model, docs) resolve against the project root, so the CLI works from any working directory.

---

Command Reference

All slash commands are tab-completable. Type /help inside the CLI for the live list. Commands are grouped below.

System

Command	Syntax	Description
Help	/help (also /?)	Show the in-CLI help panel
Exit	/exit (also /quit)	Quit the assistant
Clear	/clear	Clear conversation history
Setup	/setup	Run the first-time setup wizard (dirs + config)
Reload	/reload	Reload the model and search indexes

/help
/exit
/clear
/setup
/reload

Chat & Memory

Command	Syntax	Description
Memory list	/memory	List all stored facts (with IDs, source, tags)
Memory add	/memory add <fact>	Store a fact manually
Memory delete	/memory del <id>	Delete a fact by its ID
Memory search	/memory search <query>	Keyword + semantic search over facts
Memory auto	`/memory auto <on	off>`

Plain text messages are normal chat. Prefix a line with remember to store a fact directly. Auto-memory (on by default) extracts durable facts from ordinary sentences ("I like Python", "my project is…").

/memory
/memory add I prefer dark mode in editors
/memory del 07a1fc32f97c481a
/memory search which language do i like
/memory auto off
remember I work on a Rust project called Ferry

See Natural-Language Memory for the remember / note: / actually forms.

Documents

Command	Syntax	Description
Ingest	/ingest	Ingest every file in data/docs
Ingest path	/ingest <path>	Ingest a specific file or directory
Reload	/reload	Reload model and indexes (after editing docs)

After /ingest, the hybrid (BM25 + FAISS) retriever is rebuilt and saved automatically — there is no separate "build index" step.

/ingest
/ingest data/docs/notes/
/ingest my-standalone-file.md

Retrieval relevance gate: results below retrieval.similarity_threshold (0.3) are filtered out, so irrelevant queries return nothing instead of a junk citation.

Models

Command	Syntax	Description
Model info	/model	Show current model path, context, threads, params + installed list
Load model	/model <path>	Load a different GGUF (absolute or name under models/)
Models picker	`/models [n	name]`
Download	`/download-model <url	repo:file> [name.gguf]`
Threads show	/threads	Show current CPU thread count
Threads set	/threads <n>	Set thread count (reloads model; re-applies active LoRA)

/model
/model models/Phi-3-mini-4k-instruct-q4.gguf
/models
/models 2
/download-model https://huggingface.co/owner/model/resolve/main/m.q4_k_m.gguf
/download-model owner/model.q4_k_m.gguf
/threads
/threads 6

Model path resolution: absolute paths are used as-is; relative names always resolve inside the canonical models/ directory (never the repo root or a system path). Changing threads reloads the model and reapplies any active LoRA.

LoRA

Command	Syntax	Description
List	/lora	List LoRA adapters found in lora/
Load	/lora <path>	Load a LoRA adapter (.gguf or */adapter*.bin)
Unload	/lora none	Unload the active adapter

/lora
/lora lora/my-adapter
/lora none

Images

Command	Syntax	Description
Text-to-image	/image <prompt>	Generate an image from a text prompt (local SD)
Image-to-image	/img2img <path> <prompt>	Transform an image (photo → anime, etc.)
Image-to-image (alias)	/image2image <path> <prompt>	Same as /img2img
Select model	/img-model	List & select the Stable Diffusion checkpoint

Both /image and /img2img ask "Use recommended settings? [Y/n]" after you enter the prompt. Press Enter (or y) to use the defaults; n opens an interactive tweaker:

Setting	Range	Default	Notes
Steps	1–150	25	Diffusion steps
Guidance	1.0–30.0	7.5	Classifier-free guidance scale
Max resolution	256–2048	768	Width = height (px)
Strength	0.0–1.0	0.6	img2img only — 0 = keep original, 1 = fully redo

/img2img accepts the path first (/img2img img.png anime style) or last (/img2img anime style img.png); with no args it prompts interactively.

/image a serene mountain lake at sunset, cinematic
/img2img photo.jpg in the style of a 1980s anime
/image2image photo.jpg oil painting of a harbor
/img-model

Audio

Command	Syntax	Description
Generate	/audio <prompt>	Generate audio/music (MusicGen or ACE-Step)
Settings shortcut	/audio --sec N <prompt>	Skip the panel, set clip length to N seconds
Select model	`/audio-model [n	name

/audio opens a settings panel (clip length in seconds + guidance scale) unless you use the --sec N shortcut. Recommended defaults: MusicGen guidance 3.0; ACE-Step guidance 7.5 / steps 100; clip length 8.0s (configurable). Press n to tweak:

Setting	Range	Default	Backend
Clip length (sec)	1.0–30.0	8.0	both
Guidance	1.0–15.0	3.0 (MG) / 7.5 (ACE)	both
Steps	1–200	100	ACE-Step only

/audio-model lists local models in models/audio/ (and models/). Pick by number/name, type musicgen or acestep to use that backend's default, or type any HuggingFace repo id (owner/model). Unsupported entries (a lone weight file, a folder without config.json) are flagged ✗ with a reason.

/audio a calm lo-fi beat
/audio --sec 12 an epic orchestral score
/audio-model
/audio-model acestep
/audio-model facebook/musicgen-medium

CPU caveat: MusicGen-small handles short clips reasonably; ACE-Step (3.5B) is very slow and memory-hungry on CPU — a hardware limit, not a bug. If a generation produces near-silent audio, the CLI warns (the checkpoint is likely broken/untrained rather than a code error).

Agent

Command	Syntax	Description
Toggle	`/agent [on	off]`

/agent on
/agent off
/agent          # show current status

When on, chat is routed through a tool-use loop. Available local tools: calculator, read_file, search_docs, open_app, open_file, shell. OS actions (open_app / open_file) and the shell tool are configurable in AgentConfig.

Config & Stats

Command	Syntax	Description
Config	/config	Print the active configuration (YAML)
Stats	/stats	Show session stats (facts, chunks, LoRA, last gen speed)

/config
/stats

---

Natural-Language Memory

Beyond /memory add, the assistant captures facts from ordinary chat:

• remember I prefer Python over JavaScript — store a fact explicitly.
• note: my API key is in .env — shorthand for "remember".
• actually, the meeting is at 3pm not 2pm — correction; updates/adds a fact.
• Automatic: first-person statements and preferences ("I like Rust", "my project is…") are captured in the background when auto-memory is enabled.

Facts are stored in data/memory/user_facts.json and searched with a keyword + (optional) semantic fallback.

---

Configuration

config.yaml lives at the project root. Relative paths resolve to the project root. Key defaults:

llm:
  model_path: models/llama-3.2-3b-instruct-q4_k_m.gguf
  n_ctx: 32768            # context window (tokens)
  n_threads: 0           # 0 = auto (CPU count - 1)
  n_gpu_layers: 0        # keep 0 for CPU-only
  temperature: 0.7
  max_tokens: 2048

embedding:
  model_name: sentence-transformers/all-MiniLM-L6-v2
  device: cpu

reranker:
  enabled: true
  device: cpu

retrieval:
  top_k: 5               # candidates retrieved before rerank
  top_k_rerank: 3        # results kept after rerank
  similarity_threshold: 0.3   # relevance gate (filters junk hits)

memory:
  file_path: data/memory/user_facts.json
  auto_memory: true      # capture durable facts from chat
  embed_facts: true      # semantic search over facts (falls back to keyword)

ingestion:
  docs_directory: data/docs

vector_store:
  persist_directory: data/index/chromadb

audio:
  backend: musicgen                  # "musicgen" | "acestep"
  musicgen_model: facebook/musicgen-small
  acestep_model: ACE-Step/ACE-Step-3.5B
  duration: 8.0                      # default clip length (sec)
  acestep_steps: 100
  acestep_guidance: 7.5
  output_dir: data/audio

agent:
  enabled: false
  shell_enabled: true
  open_enabled: true

Most fields can also be set via environment variables with a prefix, e.g. AUDIO_BACKEND=acestep, LLM_N_THREADS=6.

---

CPU Optimization

Thread count

Auto-detected by default (CPU cores - 1). Override with /threads <n> or --threads N. More threads = faster generation, up to your physical core count.

Model selection for older CPUs

Intel/AMD x86 CPUs are auto-detected at startup. If your machine lacks AVX2, the CLI selects a compatible llama.cpp build automatically.

CPU Generation	Recommended Models
3rd–4th Gen (Haswell/Skylake)	1B–3B Q4_K_M (~1.5–2GB)
5th–6th Gen	3B–7B Q4_K_M (~2–4GB)
7th Gen+	7B–13B Q4_K_M (~4–8GB)

Memory budget (8 GB RAM example)

• Model (3B Q4): ~2 GB
• Embeddings: ~200 MB
• FAISS index: ~100 MB per 10k chunks
• KV cache: ~2 MB per 1k context
• Total for 8 GB RAM: use a 3B model and keep n_ctx modest.

---

LoRA Fine-Tuning (Optional)

For deeper adaptation beyond RAG:

# 1. Prepare training data (JSONL, messages format)
cat > data/train.jsonl << 'EOF'
{"messages": [{"role": "user", "content": "What is our API?"}, {"role": "assistant", "content": "Our API is REST-based..."}]}
{"messages": [{"role": "user", "content": "How to deploy?"}, {"role": "assistant", "content": "Use docker-compose..."}]}
EOF

# 2. Train a LoRA adapter
python -m lexical_llm.train_lora \
  --data data/train.jsonl \
  --model models/llama-3.2-3b-instruct-q4_k_m.gguf \
  --output lora/my-adapter \
  --epochs 3
# (alias: lexical-train ...)

# 3. Use it in the CLI
/lora lora/my-adapter

---

Project Structure

lexical-llm/
├── config.yaml                # Configuration (relative paths → project root)
├── requirements.txt           # Python dependencies
├── pyproject.toml             # Package metadata + entry points
├── data/
│   ├── docs/                 # Markdown files to ingest
│   ├── index/                # Vector indexes (FAISS + BM25 + Chroma)
│   ├── memory/               # User facts (JSON)
│   ├── images/               # Generated images
│   └── audio/                # Generated audio (.wav)
├── models/                   # GGUF models
│   ├── audio/                # Local audio model folders / weights
│   └── image/                # Local image checkpoints
├── lora/                     # LoRA adapters
├── src/lexical_llm/
│   ├── __init__.py
│   ├── config.py             # Configuration management
│   ├── model.py              # llama.cpp wrapper
│   ├── ingest.py             # Markdown parsing / chunking / embeddings
│   ├── memory.py             # User memory store
│   ├── retrieval.py          # Hybrid search (BM25 + FAISS + rerank)
│   ├── imagegen.py           # Stable Diffusion (txt2img / img2img)
│   ├── audiogen.py           # MusicGen / ACE-Step
│   ├── agent.py              # Local tool-use agent
│   ├── cli.py                # Interactive CLI
│   ├── train_lora.py         # LoRA training
│   ├── scripts/ingest.py     # Standalone ingest entry point
│   └── utils.py              # Utilities
└── tests/                    # Unit tests

---

Running Tests

pytest tests/ -v

Tests use mocked models/embedders (no downloads). Audio and image tests exercise the CLI/settings paths with fake pipelines so they run without GPU or network.

---

Troubleshooting

Model fails to load

• Check model_path in config.yaml (or your --model argument).
• Confirm the GGUF file is valid.
• A relative model name must exist under models/.

Out of memory

• Reduce n_ctx (default 32768 — large for low-RAM machines; 4096–8192 is often plenty).
• Use a smaller model (1B–3B) or heavier quantization (Q3_K / Q2_K).
• Close other applications.

Slow generation

• Increase /threads up to your physical core count.
• Use a smaller context window.
• Lower the model size or quantization level.
• Audio: ACE-Step is inherently slow on CPU; prefer MusicGen-small for quick clips.

Silent audio output

• The model checkpoint is likely broken, partially merged, or untrained — the CLI warns when peak amplitude is near zero. Re-download a complete model folder (config.json + weights + tokenizer for MusicGen).

Import errors

pip install -r requirements.txt --force-reinstall

Do not bump torch / transformers / diffusers from the pinned versions.

Agent tools not working

• OS actions and the shell tool are gated by agent.shell_enabled / agent.open_enabled in config.yaml.

---

License & Acknowledgments

MIT License — see the LICENSE file for details.

• llama.cpp — CPU inference engine
• sentence-transformers — embeddings
• ChromaDB — vector store
• FAISS — dense similarity search
• rank-bm25 — sparse BM25 search
• diffusers — image generation
• transformers — MusicGen / ACE-Step
• rich + prompt_toolkit — the terminal UI