quaynor 2.0.0

Local GGUF inference, tool calling, and streaming chat for Python (Quaynor bindings).

Quaynor

Run LLMs locally and efficiently on any device

Quaynor is a lightweight, fast inference engine that makes it simple to run open-source LLMs directly inside your Python applications.

Key Features

• Run locally, offline, for free - No API keys or cloud services required
• Fast, simple tool calling - Just pass normal Python functions
• Reliable tool execution - Automatically derives grammar from function signatures
• Infinite conversations - Conversation-aware preemptive context shifting prevents mid-conversation crashes
• GPU accelerated — Vulkan (Linux/Windows) and Metal (macOS/iOS), where supported
• Thousands of compatible models - Works with any LLM in GGUF format
• Powered by llama.cpp - Built on the proven llama.cpp engine

Quick Start

from quaynor import Chat

chat = Chat('./model.gguf')
response = chat.ask('Hello world?').completed()
print(response)

Installation

pip install quaynor

Documentation

Full documentation: https://www.quaynor.site/

License

MIT — see the repository LICENSE file.