llmfastdev
A Python client for interacting with llamafile local LLM instances.
Installation
pip install llmfastdev
Usage
from llmfastdev import LlamafileClient
# Initialize the client
client = LlamafileClient("http://localhost:8080")
# Check if server is running
if client.health_check():
print("Server is running!")
# Generate a completion
response = client.complete("Hello, how are you?", max_tokens=50)
print(response)
# Use chat format
messages = [
{"role": "user", "content": "What is the capital of France?"}
]
response = client.chat(messages, max_tokens=50)
print(response)
# Stream completion
for token in client.stream_completion("Tell me a story", max_tokens=100):
print(token, end="", flush=True)
Features
- Simple HTTP client for llamafile servers
- Support for completion and chat endpoints
- Streaming support
- Context manager support
- Health check functionality
- Model information retrieval
Requirements
- Python >= 3.7
- requests >= 2.25.0
License
MIT
1 maintainer
Metadata
1 maintainer
Release files for llmfastdev 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| llmfastdev-0.0.1.tar.gz | 4.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| llmfastdev-0.0.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 9.1 kB
Release files / llmfastdev-0.0.1.tar.gz
| Download URL | llmfastdev-0.0.1.tar.gz |
|---|---|
| Size | 4.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
557fcebbc0137ed3815eb956d57df23f36a145602f3b592d3c87313b574a0b3d
|
|
BLAKE2b-256 checksum How to use checksums |
ef97c1f7d2a808798f34e4018c8c8e7f995b10cb1d6970b3a5f7e52b5e6045bb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.11
|
Release files / llmfastdev-0.0.1-py3-none-any.whl
| Download URL | llmfastdev-0.0.1-py3-none-any.whl |
|---|---|
| Size | 4.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5459e1ae93fe53c22d92676cf697c29e7777b1523af03c87118514b5bc4a2d5e
|
|
BLAKE2b-256 checksum How to use checksums |
4a88744adbc8bf4eec080bdc63ff7549d2194732bc98369de15245dd90d6bfdb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.11
|
