leeroy-py 0.1.0

Leeroy

Overview · Features · Modes · Requirements · LLM · Installation · Run · Configuration · Architecture · Capabilities · Data ·

Leeroy is a Python application for local language-model inference, retrieval-augmented generation, semantic search, prompt engineering, and SQLite-backed data management. It is designed for federal analysts, technical users, and data-science workflows that benefit from local execution, durable prompt storage, document-grounded question answering, and controlled analytical tooling.

Leeroy uses an optional local GGUF model through llama.cpp, supports document retrieval with sentence-transformers, persists chat history and prompts in SQLite, and provides a Streamlit user interface for text generation, document Q&A, semantic indexing, prompt administration, and database operations.

🎥 Demo

☁️ Cloud

🧠 Custom LLM

Leeroy is designed to run against a local GGUF model, commonly a fine-tuned Llama-family model quantized for efficient local inference.

Leeroy-3B-Instruct.Q4_K_M.gguf

🧰 Overview

Leeroy is a local-first analytical assistant built around a Streamlit interface and a configurable local GGUF model. The application checks whether the configured model file exists, lazily loads the model only when enabled and available, and routes text-generation and document-question-answering turns through a shared prompt builder.

The application is intentionally simple and durable:

• The model is stored outside the repository.
• The app reads the model path from configuration.
• Chat history, embeddings, and prompts are persisted in SQLite.
• Document Q&A uses local text extraction, chunking, embeddings, and retrieval.
• Data Management provides practical SQLite inspection and administration tools.

✨ Features

• Local LLM inference through llama.cpp and a configurable GGUF model path.
• Lazy model loading so environments without the model can still load the Streamlit UI.
• Text Generation mode with streaming output and adjustable inference parameters.
• Document Q&A mode with upload-based document retrieval and local RAG prompt construction.
• Semantic Search mode for building an embedding index from uploaded documents.
• Prompt Engineering mode backed by the local SQLite Prompts table.
• Data Management mode for importing, browsing, editing, profiling, visualizing, and querying SQLite data.
• System Instructions editor with reusable prompt-template loading.
• SQLite persistence for chat history, prompt records, embeddings, and imported tabular data.
• Document text extraction using PyMuPDF for PDFs and defensive text decoding for text-like files.
• Vector retrieval using sentence-transformers, sqlite-vec when available, and cosine-similarity fallback.
• Interactive visualizations through Plotly Express.
• Fixed footer status bar showing active mode, temperature, Top-P, Top-K, penalties, token limits, context size, thread count, semantic status, and loaded document counts.

🧩 Application Modes

🛠️ Requirements

🧊 Local LLM

Leeroy uses optional local LLM support. The model is loaded only when local LLM support is enabled and the configured GGUF file exists.

The relevant configuration pattern is:

ENABLE_LOCAL_LLM = True
MODEL_PATH = "C:/path/to/Leeroy-3B-Instruct.Q4_K_M.gguf"
DEFAULT_CTX = 4096
CORES = 4

The model-loading path uses llama_cpp.Llama with:

Recommended model locations include:

• A local models/ or llm/ folder.
• An LM Studio model cache.
• An external drive.
• A shared read-only model directory.

Example Windows path:

C:\Users\<you>\source\llm\lmstudio\lmstudio-community\leeroy-jankins\leeroy\Leeroy-3B-Instruct.Q4_K_M.gguf

📦 Installation

1. Clone the Repository

git clone https://github.com/is-leeroy-jenkins/Leeroy.git
cd Leeroy

2. Create and Activate a Virtual Environment

Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1

Command Prompt:

python -m venv .venv
.venv\Scripts\activate.bat

macOS / Linux:

python -m venv .venv
source .venv/bin/activate

3. Install Dependencies

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

⚙️ Configuration

Leeroy is configured primarily through config.py.

Common configuration values include:

Optional environment-variable pattern:

setx LEEROY_LLM_PATH "C:\path\to\Leeroy-3B-Instruct.Q4_K_M.gguf"

Then bind that value inside config.py if the project uses environment-based model path resolution.

🚀 Running the Streamlit Application

From the project root:

streamlit run app.py

Once running, the application is available at:

http://localhost:8501

💬 Text Generation

The Text Generation mode provides streaming local inference through the shared run_llm_turn() pipeline.

Control groups include:

The text-generation workflow is:

1. User enters a chat prompt.
2. The app stores the user message in SQLite.
3. The prompt builder combines system instructions, optional semantic context, document context, and chat history.
4. llama.cpp streams the response into the Streamlit chat message.
5. The assistant message is saved to SQLite.

📚 Document Q&A

The Document Q&A mode supports local retrieval-augmented generation.

Supported upload types include:

• pdf
• txt
• docx

The workflow includes:

1. Upload one or more documents.
2. Store active document names and bytes in Streamlit session state.
3. Preview the first active document when supported.
4. Extract text from PDF bytes using PyMuPDF.
5. Chunk extracted text.
6. Generate embeddings with sentence-transformers.
7. Store vectors in sqlite-vec if available.
8. Fall back to in-memory cosine similarity when vector-table support is unavailable.
9. Retrieve the top relevant document chunks.
10. Build a document-grounded prompt.
11. Stream the answer from the local model.

🔍 Semantic Search

The Semantic Search mode builds a reusable local embedding index.

Semantic Search supports:

• Uploading multiple files.
• Chunking decoded text.
• Embedding chunks with sentence-transformers.
• Storing chunks and vector blobs in the SQLite embeddings table.
• Enabling semantic context through the use_semantic session-state flag.

When semantic context is enabled, Text Generation can retrieve similar chunks from the embedding store and inject them into the prompt.

📝 Prompt Engineering

The Prompt Engineering mode manages reusable prompt records in the SQLite Prompts table.

Prompt records include:

Prompt Engineering supports:

• Search by prompt name or text.
• Sort by prompt fields.
• Sort direction selection.
• Pagination.
• Jump-to-ID navigation.
• Prompt row selection.
• Editing existing prompts.
• Creating new prompts.
• Deleting prompts.
• Clearing the current selection.
• Cascading selected prompt content into system instructions.

🏛️ Data Management

The Data Management mode provides a SQLite administration and exploration interface.

Tabs include:

Administrative operations include:

• Table profiling.
• Table dropping with confirmation.
• Index creation.
• Custom table creation.
• Schema viewing.
• Row-count metrics.
• Index listing.
• Add column.
• Rename column.
• Rename table.
• Drop column.

The SQL console allows read-only SQL workflows and blocks unsafe operations such as:

• INSERT
• UPDATE
• DELETE
• DROP
• ALTER
• CREATE
• ATTACH
• DETACH
• VACUUM
• REPLACE
• TRIGGER

🧩 Design and Architecture

Leeroy uses a local-first Streamlit architecture:

Architecture diagram:

┌──────────────────────────────────────────────┐
│                Leeroy Streamlit App          │
│                                              │
│  - Text Generation                           │
│  - Document Q&A                              │
│  - Semantic Search                           │
│  - Prompt Engineering                        │
│  - Data Management                           │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────┐
│              Shared Prompt Pipeline          │
│                                              │
│  - System instructions                       │
│  - Chat history                              │
│  - Semantic context                          │
│  - Document excerpts                         │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────┐
│             Local llama.cpp Model            │
│                                              │
│  - GGUF model path                           │
│  - Context window                            │
│  - CPU threads                               │
│  - Streaming output                          │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────┐
│              SQLite Persistence              │
│                                              │
│  - chat_history                              │
│  - embeddings                                │
│  - Prompts                                   │
│  - imported data tables                      │
└──────────────────────────────────────────────┘

💻 Capabilities

📁 File Organization

🧪 Example Usage

Local LLM Loading Pattern

from llama_cpp import Llama

llm = Llama(
    model_path="C:/path/to/Leeroy-3B-Instruct.Q4_K_M.gguf",
    n_ctx=4096,
    n_threads=4,
    n_batch=512,
    verbose=False
)

Sentence Embedding Pattern

from sentence_transformers import SentenceTransformer

embedder = SentenceTransformer("all-MiniLM-L6-v2")
vectors = embedder.encode([
    "Federal analysts need reliable local retrieval over policy documents."
])

print(vectors.shape)

SQLite Prompt Query

import sqlite3

with sqlite3.connect("stores/sqlite/Data.db") as conn:
    rows = conn.execute(
        "SELECT PromptsId, Caption, Name, Version FROM Prompts ORDER BY PromptsId DESC"
    ).fetchall()

print(rows)

Streamlit Launch

streamlit run app.py

🧮 Runtime Parameters

Leeroy exposes the following active runtime parameters in the UI and footer:

🧰 Troubleshooting

🚀 Application Badges

📝 License

Leeroy is published under the MIT License.

PyPI application

python -m pip install leeroy-py
leeroy

leeroy opens the tabless Windows WebView2 window by default; leeroy-browser opens standard browser mode. Python 3.11–3.12 is supported. The distribution is leeroy-py; the import is leeroy. The GGUF model is supplied separately. Set LEEROY_LLM_PATH to its absolute path and ENABLE_LOCAL_LLM=true before launching local inference. See installation and publication.