LocalLab Python Client
Official Python client for LocalLab - A local LLM server.
Package Name Note
While the package is installed as locallab-client (with a hyphen) via pip:
pip install locallab-client
You import it using an underscore:
from locallab_client import LocalLabClient
This follows Python's package naming convention where hyphens in package names are converted to underscores for imports.
Features
- 🚀 Async/await API
- 📊 Batch processing
- 🌊 Streaming support
- 💬 Chat completion
- 🔍 Model management
- ���� System monitoring
- 🔒 Type-safe with Pydantic
- 🌐 WebSocket support
Installation
pip install locallab-client
# or
poetry add locallab-client
Quick Start
import asyncio
from locallab_client import LocalLabClient
async def main():
# Initialize client
client = LocalLabClient({
"base_url": "http://localhost:8000",
"api_key": "your-api-key", # Optional
})
try:
# Basic generation
response = await client.generate("Hello, how are you?")
print(response.response)
finally:
await client.close()
if __name__ == "__main__":
asyncio.run(main())
Usage Examples
Text Generation
# Basic generation
response = await client.generate("Hello, how are you?")
print(response.response)
# Generation with options
response = await client.generate("Hello", {
"temperature": 0.7,
"max_length": 100,
})
# Streaming generation
async for token in client.stream_generate("Tell me a story"):
print(token, end="", flush=True)
Chat Completion
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"},
]
response = await client.chat(messages)
print(response.choices[0].message.content)
Batch Processing
prompts = [
"What is 2+2?",
"Who wrote Romeo and Juliet?",
"What is the speed of light?",
]
response = await client.batch_generate(prompts)
for i, answer in enumerate(response.responses, 1):
print(f"{i}. {answer}")
Model Management
# List available models
models = await client.list_models()
print(models)
# Load a specific model
await client.load_model("mistral-7b")
# Get current model info
current_model = await client.get_current_model()
print(current_model)
System Monitoring
# Get system information
system_info = await client.get_system_info()
print(f"CPU Usage: {system_info.cpu_usage}%")
print(f"Memory Usage: {system_info.memory_usage}%")
if system_info.gpu_info:
print(f"GPU: {system_info.gpu_info.device}")
# Check system health
is_healthy = await client.health_check()
print(is_healthy)
WebSocket Connection
# Connect to WebSocket
await client.connect_ws()
# Subscribe to messages
async def message_handler(data):
print("Received:", data)
await client.on_message(message_handler)
# Disconnect when done
await client.disconnect_ws()
API Reference
Client Configuration
class LocalLabConfig(BaseModel):
base_url: str
api_key: Optional[str] = None
timeout: float = 30.0
retries: int = 3
headers: Dict[str, str] = Field(default_factory=dict)
Generation Options
class GenerateOptions(BaseModel):
model_id: Optional[str] = None
max_length: Optional[int] = None
temperature: Optional[float] = None
top_p: Optional[float] = None
stream: bool = False
Response Types
class GenerateResponse(BaseModel):
response: str
model_id: str
usage: Usage
class ChatResponse(BaseModel):
choices: List[ChatChoice]
usage: Usage
Error Handling
The client throws typed exceptions that you can catch and handle:
try:
await client.generate("Hello")
except ValidationError as e:
print("Validation error:", e.field_errors)
except RateLimitError as e:
print(f"Rate limit exceeded. Retry after {e.retry_after}s")
except LocalLabError as e:
print(f"Error {e.code}: {e.message}")
Development
Installation
# Install dependencies
pip install -e ".[dev]"
Testing
# Run tests
pytest
# Run tests with coverage
pytest --cov=locallab
Linting
# Run linters
flake8 locallab
mypy locallab
Formatting
# Format code
black locallab
isort locallab
Contributing
- Fork the repository
- Create your feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
License
This project is licensed under the MIT License - see the LICENSE file for details.
Metadata
Release files for locallab-client 1.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| locallab_client-1.1.0.tar.gz | 15.7 kB | Without provenance | Details for locallab_client-1.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| locallab_client-1.1.0-py3-none-any.whl | Python 3 | none | any | Details for locallab_client-1.1.0-py3-none-any.whl |
Total release size: 28.8 kB
Release files / locallab_client-1.1.0.tar.gz
| Download URL | locallab_client-1.1.0.tar.gz |
|---|---|
| Size | 15.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5209ac378b13534411bc94e85a267d7b63baf7b926b1322551dcac0faffb77a4
|
|
BLAKE2b-256 checksum How to use checksums |
4cba1f1a52ef2c1e9a5fc9d48fc8c818dcf2e48d726e43040f497b3ccdf56e78
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.9.22
|
Release files / locallab_client-1.1.0-py3-none-any.whl
| Download URL | locallab_client-1.1.0-py3-none-any.whl |
|---|---|
| Size | 13.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d494de75e30104d7ce337276c44983f5ad17ee74c0d5bad8d3e1a881d0327cc2
|
|
BLAKE2b-256 checksum How to use checksums |
3dbe44ae75589070bec2ec5a8e30152d0026532c69318e0c354196c7b67719f2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.9.22
|
Security for locallab-client 1.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.