LLMlight 0.2.1

LLMlight is a Python library for ...

LLMlight

LLMlight is a Python package for running Large Language Models (LLMs) locally with minimal dependencies. It provides a simple interface to interact with various LLM models, including support for GGUF models and local API endpoints.

🌟 Key Features

• Local LLM Support: Run LLMs locally with minimal dependencies
• Multiple Model Support: Compatible with various models including:
  • Hermes-3-Llama-3.2-3B
  • Mistral-7B-Grok
  • OpenHermes-2.5-Mistral-7B
  • Gemma-2-9B-IT
• Flexible Embedding Methods: Support for multiple embedding approaches:
  • TF-IDF for structured documents
  • Bag of Words (BOW)
  • BERT for free text
  • BGE-Small
• Advanced Retrieval Methods:
  • Naive RAG with fixed chunking
  • RSE (Relevant Segment Extraction)
• PDF Processing: Built-in support for reading and processing PDF documents
• Global Reasoning: Advanced reasoning capabilities for complex queries

📚 Documentation & Resources

• Documentation
• Blog Posts
• GitHub Issues

🚀 Quick Start

Installation

# Install from PyPI
pip install LLMlight

# Install from GitHub
pip install git+https://github.com/erdogant/LLMlight

Basic Usage

from LLMlight import LLMlight

# Initialize with default settings
model = LLMlight()

# Run a simple query
response = model.prompt('What is the capital of France?', 
                    system="You are a helpful assistant.")

# Use with a local GGUF model
model = LLMlight(endpoint='path/to/your/model.gguf')
response = model.prompt('Tell me about quantum computing')

📊 Examples

1. Using with LM Studio

from LLMlight import LLMlight

# Initialize with LM Studio endpoint
model = LLMlight(endpoint="http://localhost:1234/v1/chat/completions")

# Run queries
response = model.prompt('Explain quantum computing in simple terms')

2. Validate Models

from LLMlight import LLMlight

# Initialize model
from LLMlight import LLMlight
model = LLMlight(verbose='info')

modelnames = model.get_available_models(validate=True)
print(modelnames)

3. Processing PDF Documents and Ask Questions

from LLMlight import LLMlight

# Initialize model
model = LLMlight()

# Read and process PDF
model.read_pdf('path/to/document.pdf')

# Query about the document
response = model.prompt('Summarize the main points of this document')

print(response)

4. Global Reasoning

from LLMlight import LLMlight

# Initialize model
model = LLMlight()

# Read and process PDF
model.read_pdf('path/to/document.pdf')

# Query about the document
response = model.prompt('Summarize the main points of this document', global_reasoning=True)

print(response)

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

👥 Contributors

👨‍💻 Maintainer

• Erdogan Taskesen (@erdogant)

☕ Support

This library is free and open source. If you find it useful, consider supporting its development:

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.