LLMlight 1.2.0

LLMlight is a Python library for ...

LLMlight

LLMlight is a Python package for running Large Language Models (LLMs) locally with minimal dependencies. It provides a simple interface to interact with various LLM models, including support for GGUF models and local API endpoints. ⭐️Star it if you like it⭐️

---

Key Features

Feature	Description
Local LLM Support	Run LLMs locally with minimal dependencies.
Full Prompt Control	Fine-grained control over prompts including Query, Instructions, System, Context, Response Format, Automatic formatting, Temperature, and Top P.
Single Endpoint for All Local Models	One unified endpoint to connect different local models.
Flexible Embedding Methods	Multiple embedding strategies: TF-IDF for structured documents, Bag of Words (BOW), BERT for free text, BGE-Small.
Advanced Retrieval Methods	Supports Naive RAG with fixed chunking and RSE (Relevant Segment Extraction).
Context Strategies	Advanced reasoning for complex queries using Global-reasoning and Chunk-wise approaches.
Local Memory	Video memory storage for efficient local use.
PDF Processing	Native support for reading and processing PDF documents.

---

Documentation & Resources

• Documentation
• Blog Posts
• GitHub Issues

Installation

# Install from PyPI
pip install LLMlight

Quick Start

1. Check Available Models at Endpoint

from LLMlight import LLMlight

# Initialize client
from LLMlight import LLMlight
# Initialize with LM Studio endpoint
client = LLMlight(model='mistralai/mistral-small-3.2',
                  endpoint="http://localhost:1234/v1/chat/completions")

modelnames = client.get_available_models(validate=False)
print(modelnames)

2. Basic Usage with Endpoint

from LLMlight import LLMlight

# Initialize with default settings
client = LLMlight(model='openai/gpt-oss-20b', endpoint='http://localhost:1234/v1/chat/completions')

# Run a simple query
response = client.prompt('What is the capital of France?',
                         context='The capital of France is Amsterdam.',
                         instructions='Do not argue with the information in the context. Only return the information from the context.')
print(response)
# According to the provided context, the capital of France is Amsterdam.

3. Using with LM Studio

https://erdogant.github.io/LLMlight/pages/html/Algorithm.html#get-available-llm-models

3. Query against PDF files

https://erdogant.github.io/LLMlight/pages/html/Examples.html#working-with-files-pdfs

4. Creating Local Memory Database

https://erdogant.github.io/LLMlight/pages/html/Saving%20and%20Loading.html#memory-management

Maintainer

• Erdogan Taskesen, github: erdogant
• Contributions are welcome.
• Yes! This library is entirely free but it runs on coffee! :) Feel free to support with a Coffee.