jimi-py 0.1.0

Jimi

Overview • Features • Modes • Requirements • LLM • Installation • Run • Configuration • Architecture • Capabilities • Data •

Jimi is a Python and Streamlit application for local language-model inference, retrieval-augmented generation, semantic search, prompt engineering, and SQLite-backed data management. It is designed for federal analysts, technical users, and data-science workflows that benefit from local execution, durable prompt storage, document-grounded question answering, semantic retrieval, and controlled analytical tooling.

Jimi uses an optional local GGUF model through llama.cpp, supports document retrieval with sentence-transformers, persists chat history and prompts in SQLite, and provides a Streamlit user interface for text generation, document Q&A, semantic indexing, prompt administration, and database operations.

☁️ Cloud

🧰 Overview

Jimi is a local-first analytical assistant built around a Streamlit interface and a configurable local GGUF model. The application checks whether the configured model file exists, lazily loads the model only when available, and routes text-generation and document-question-answering turns through a shared prompt builder.

The application is intentionally simple and durable:

• The model is stored outside the repository.
• The app reads the model path from config.py.
• Chat history, embeddings, and prompts are persisted in SQLite.
• Document Q&A uses local text extraction, chunking, embeddings, and retrieval.
• Semantic Search builds a reusable SQLite-backed embedding index.
• Data Management provides practical SQLite inspection, import, query, visualization, and schema tools.

✨ Features

• Local LLM inference through llama.cpp and a configurable GGUF model path.
• Deferred dependency resolution so optional local model and embedding dependencies do not prevent the Streamlit UI from loading.
• Lazy model loading with st.cache_resource for local model and embedding resources.
• Text Generation mode with streaming output and adjustable task, reasoning, coding, response, inference, and context controls.
• Document Q&A mode with upload-based retrieval, document actions, parsing controls, local RAG prompt construction, retrieved-chunk display, and document inventory diagnostics.
• Semantic Search mode for building, querying, maintaining, and reusing a local embedding index.
• Prompt Engineering mode backed by the local SQLite Prompts table, including prompt generation, cloning, metadata, category inference, and cascade actions into Text Generation or Document Q&A.
• Data Management mode for importing, browsing, editing, profiling, visualizing, and querying SQLite data.
• System Instructions editor with reusable prompt-template loading, XML-to-Markdown conversion, preset application, and effective-prompt preview.
• SQLite persistence for chat history, prompt records, embeddings, and imported tabular data.
• Document text extraction using PyMuPDF for PDFs and defensive text decoding for text-like files.
• Vector retrieval using sentence-transformers, sqlite-vec when available, and cosine-similarity fallback.
• Interactive visualizations through Plotly Express.
• Fixed footer status bar showing active mode, temperature, Top-P, Top-K, penalties, token limits, context size, thread count, semantic status, and loaded document count.

🧩 Application Modes

🛠️ Requirements

🧠 Custom LLM

• Jimi uses optional local LLM support.

Jimi is designed to run against a local GGUF model, commonly a Gemma-family or Llama-family model quantized for efficient local inference. The repository contains application code; the model should be stored separately and referenced through configuration. The model is loaded only when the configured GGUF file exists. The local_model_available() function checks the configured model path before llama_cpp.Llama is imported and initialized.

Example model name:

Gemma-4-E4B-it.gguf

The relevant configuration pattern is:

MODEL_PATH = "C:/path/to/Gemma-4-E4B-it.gguf"
DEFAULT_CTX = 4096
CORES = 4
MODES = [
    "Text Generation",
    "Document Q&A",
    "Semantic Search",
    "Prompt Engineering",
    "Data Management"
]

The model-loading path uses llama_cpp.Llama with:

Recommended model locations include:

• A local models/ or llm/ folder.
• An LM Studio model cache.
• An external drive.
• A shared read-only model directory.

Example Windows path:

C:\Users\<you>\source\llm\lmstudio\lmstudio-community\leeroy-jankins\jimi\Gemma-4-E4B-it.gguf

📦 Installation

1. Clone the Repository

git clone https://github.com/is-leeroy-jenkins/Jimi.git
cd Jimi

2. Create and Activate a Virtual Environment

Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1

Command Prompt:

python -m venv .venv
.venv\Scripts\activate.bat

macOS / Linux:

python -m venv .venv
source .venv/bin/activate

3. Install Dependencies

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

4. Install llama-cpp-python

python -m pip install llama-cpp-python

⚙️ Configuration

Jimi is configured primarily through config.py.

Common configuration values include:

Optional environment-variable pattern:

setx JIMI_LLM_PATH "C:\path\to\Gemma-4-E4B-it.gguf"

Then bind that value inside config.py if the project uses environment-based model path resolution.

🚀 Running the Streamlit Application

From the project root:

streamlit run app.py

Once running, the application is available at:

http://localhost:8501

💬 Text Generation

The Text Generation mode provides streaming local inference through the shared run_llm_turn() pipeline.

Control groups include:

The text-generation workflow is:

1. User enters a chat prompt.
2. The app stores the user message in SQLite.
3. The prompt builder combines system instructions, task-specific instructions, optional semantic context, optional document context, and chat history.
4. llama.cpp streams the response into the Streamlit chat message.
5. The assistant message is saved to SQLite.

📚 Document Q&A

The Document Q&A mode supports local retrieval-augmented generation.

Supported upload types include:

• pdf
• txt
• docx

Document Q&A controls include:

The workflow includes:

1. Upload one or more documents.
2. Store active document names and bytes in Streamlit session state.
3. Preview the first active document when supported.
4. Extract text from PDF bytes using PyMuPDF or decode text-like files defensively.
5. Build document inventory rows with size, extracted text length, chunk count, and loaded status.
6. Chunk extracted text using configurable chunk size and overlap.
7. Generate embeddings with sentence-transformers.
8. Store vectors in sqlite-vec if available and enabled.
9. Fall back to in-memory cosine similarity when vector-table support is unavailable and fallback is enabled.
10. Retrieve the top relevant document chunks.
11. Build a document-grounded prompt using the selected document action.
12. Stream the answer from the local model.

🔍 Semantic Search

The Semantic Search mode builds and queries a reusable local embedding index.

Semantic Search supports:

• Uploading multiple pdf, txt, or docx files.
• Configuring chunk size and chunk overlap.
• Clearing or appending to the existing index.
• Embedding chunks with sentence-transformers.
• Storing chunks and vector blobs in the SQLite embeddings table.
• Querying the embedding index with Top-K and minimum-similarity controls.
• Selecting result rows in a Streamlit data editor.
• Sending selected chunks to Text Generation.
• Sending selected chunks to Document Q&A.
• Saving selected chunks as shared prompt context.
• Deleting the index.
• Recomputing diagnostics.
• Clearing query results.

When semantic context is enabled, Text Generation can retrieve similar chunks from the embedding store and inject them into the prompt.

📝 Prompt Engineering

The Prompt Engineering mode manages reusable prompt records in the SQLite Prompts table.

Prompt records include:

Prompt Engineering supports:

• Search by caption, name, or text.
• Prompt category inference.
• Prompt category filtering.
• Sort field and sort direction selection.
• Pagination.
• Jump-to-ID navigation.
• Prompt row selection.
• Prompt editing.
• Prompt creation.
• Prompt deletion.
• Prompt cloning.
• Starter prompt generation.
• Local-model prompt-template drafting.
• Cascading selected prompts into shared Text Generation settings.
• Cascading selected prompts into shared Document Q&A settings.

Supported prompt categories include:

• General Chat.
• Reasoning.
• Coding.
• Translation.
• Summarization.
• Extraction.
• Document Extraction.
• OCR.
• Audio.
• JSON Output.

🏛️ Data Management

The Data Management mode provides a SQLite administration and exploration interface.

Tabs include:

Administrative operations include:

• Table profiling.
• Table dropping with confirmation.
• Index creation.
• Custom table creation.
• Schema viewing.
• Row-count metrics.
• Index listing.
• Add column.
• Rename column.
• Rename table.
• Drop column.

The SQL console allows read-only SQL workflows and blocks unsafe operations such as:

• INSERT
• UPDATE
• DELETE
• DROP
• ALTER
• CREATE
• ATTACH
• DETACH
• VACUUM
• REPLACE
• TRIGGER

🧩 Design and Architecture

Jimi uses a local-first Streamlit architecture:

Architecture diagram:

┌──────────────────────────────────────────────┐
│                 Jimi Streamlit App           │
│                                              │
│  - Text Generation                           │
│  - Document Q&A                              │
│  - Semantic Search                           │
│  - Prompt Engineering                        │
│  - Data Management                           │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────┐
│              Shared Prompt Pipeline          │
│                                              │
│  - System instructions                       │
│  - Task-specific instructions                │
│  - Chat history                              │
│  - Semantic context                          │
│  - Document excerpts                         │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────┐
│             Local llama.cpp Model            │
│                                              │
│  - GGUF model path                           │
│  - Context window                            │
│  - CPU threads                               │
│  - Streaming output                          │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
┌──────────────────────────────────────────────┐
│              SQLite Persistence              │
│                                              │
│  - chat_history                              │
│  - embeddings                                │
│  - Prompts                                   │
│  - imported data tables                      │
└──────────────────────────────────────────────┘

💻 Capabilities

📁 File Organization

🧪 Example Usage

Local LLM Loading Pattern

from llama_cpp import Llama

llm = Llama(
    model_path="C:/path/to/Gemma-4-E4B-it.gguf",
    n_ctx=4096,
    n_threads=4,
    n_batch=512,
    verbose=False
)

Sentence Embedding Pattern

from sentence_transformers import SentenceTransformer

embedder = SentenceTransformer("all-MiniLM-L6-v2")
vectors = embedder.encode([
    "Federal analysts need reliable local retrieval over policy documents."
])

print(vectors.shape)

SQLite Prompt Query

import sqlite3

with sqlite3.connect("stores/sqlite/Data.db") as conn:
    rows = conn.execute(
        "SELECT PromptsId, Caption, Name, Version FROM Prompts ORDER BY PromptsId DESC"
    ).fetchall()

print(rows)

Streamlit Launch

streamlit run app.py

🧮 Runtime Parameters

Jimi exposes the following active runtime parameters in the UI and footer:

🧰 Troubleshooting

🚀 Application Badges

📝 License

Jimi is published under the MIT License.

PyPI application

python -m pip install jimi-py
jimi

jimi opens the tabless Windows WebView2 window by default. jimi-browser opens standard browser mode. Python 3.11–3.12 is supported. The distribution is jimi-py; the import is jimi. Set GEMINI_API_KEY before using Gemini features. Windows requires WebView2 Runtime. See installation and publication.