Convlit 0.1.1

Chat with local GGUF models (no llama.cpp).

Convlit (GGUF Chat Runner — No llama.cpp)

Chats with a local .gguf model without using llama.cpp / llama-cpp-python.

This runner:

• Parses GGUF directly (mmap)
• Keeps the GGUF file mapped while running (so tensors can safely share the mmap)
• Dequantizes common GGML K-quants (Q4_K, Q5_K, Q6_K) and IQ4_XS
• Computes RoPE frequencies from GGUF metadata (llama.rope.freq_base / llama.rope.dimension_count)
• Runs a minimal Llama-3.x style model in PyTorch (CPU)

Setup

Install (PyPI)

python -m pip install convlit

Install (from source)

python -m pip install -r requirements.txt

Run

convlit --model .\ai.gguf

In-chat commands:

• /help, /reset, /system <text>, /save <path>, /exit