Minimal GGUF transformer runtime and OpenAI-compatible local chat server.
Project description
tinyq4
tinyq4 is a minimal Rust GGUF transformer runtime with an OpenAI-compatible
local chat completions server.
Install
pip install tinyq4
Run
Pass a GGUF model path explicitly:
tinyq4 /path/to/model.gguf --server 18081
Then check the server:
curl http://127.0.0.1:18081/health
The server exposes:
GET /healthGET /v1/modelsPOST /v1/chat/completions
Development
Build from source with Cargo:
cargo build --release
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tinyq4-0.1.0.tar.gz
(45.6 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file tinyq4-0.1.0.tar.gz.
File metadata
- Download URL: tinyq4-0.1.0.tar.gz
- Upload date:
- Size: 45.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a7fa7a0a66a6b80257617c4a4e05393560e9a95018ed363cc8a68b4e0f737f6d
|
|
| MD5 |
fb8415a626daf97c56541b7c2d1f4ff0
|
|
| BLAKE2b-256 |
0a5b60416393a72b44179b7d6a631473c68cf4505c7eaf6d1f065b407807c0f5
|
File details
Details for the file tinyq4-0.1.0-py3-none-manylinux_2_34_x86_64.whl.
File metadata
- Download URL: tinyq4-0.1.0-py3-none-manylinux_2_34_x86_64.whl
- Upload date:
- Size: 853.5 kB
- Tags: Python 3, manylinux: glibc 2.34+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
33ffa2663656f3616a1e7fe8bec24b27f997881fddb6be2ed0a5663a3713706e
|
|
| MD5 |
b8cbc505b5e005ba70d2aa9de73a59e6
|
|
| BLAKE2b-256 |
78ed7a2739015030f44ff10da28d020287f8f563c0c018ff0557579dae4e6ae6
|
