Quaynor
Run LLMs locally and efficiently on any device
Quaynor is a lightweight, fast inference engine that makes it simple to run open-source LLMs directly inside your Python applications.
Key Features
- Run locally, offline, for free - No API keys or cloud services required
- Fast, simple tool calling - Just pass normal Python functions
- Reliable tool execution - Automatically derives grammar from function signatures
- Infinite conversations - Conversation-aware preemptive context shifting prevents mid-conversation crashes
- GPU accelerated — Vulkan (Linux/Windows) and Metal (macOS/iOS), where supported
- Thousands of compatible models - Works with any LLM in GGUF format
- Powered by llama.cpp - Built on the proven llama.cpp engine
Quick Start
from quaynor import Chat
chat = Chat('./model.gguf')
response = chat.ask('Hello world?').completed()
print(response)
Installation
pip install quaynor
Documentation
Full documentation: https://www.quaynor.site/
License
MIT — see the repository LICENSE file.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
quaynor-2.0.0.tar.gz
(2.0 MB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file quaynor-2.0.0.tar.gz.
File metadata
- Download URL: quaynor-2.0.0.tar.gz
- Upload date:
- Size: 2.0 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
maturin/1.13.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
df2fb9ebecaf098b3820fe17a4cff3f2c3cfddde99fd641098a53167c7583d4f
|
|
| MD5 |
d7c7567825f18a1c34841070d66d9595
|
|
| BLAKE2b-256 |
f7d4888e8016955d3c17562766d5158187e5ae4be328920b2a2163ea836764a7
|
File details
Details for the file quaynor-2.0.0-cp38-abi3-macosx_11_0_arm64.whl.
File metadata
- Download URL: quaynor-2.0.0-cp38-abi3-macosx_11_0_arm64.whl
- Upload date:
- Size: 9.9 MB
- Tags: CPython 3.8+, macOS 11.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via:
maturin/1.13.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c9121895f4e0929d2242f6294b7e269fb0103e2f73320f8f50c740344466aa3e
|
|
| MD5 |
eb6c476e2c374c0b44a74ff3e099738c
|
|
| BLAKE2b-256 |
c2eefefa160fbb2cdf80946911332039cba7c1744c5c52dadb7c0bd79b08ebe4
|
