zerollm-kit 0.1.3

Zero setup, zero config — the easiest Python API for local LLMs on any hardware

Zero setup. Zero config. Local LLMs on any hardware.

---

What is ZeroLLM?

One pip install. Auto-detects your hardware. Downloads the right model. You're chatting in 3 lines of Python.

from zerollm import Chat

bot = Chat("Qwen/Qwen3-0.6B")
print(bot.ask("What is the capital of France?"))

That's it. No config files, no model format headaches, no GPU drivers to manage.

Install

pip install zerollm-kit

Quick Start

Chat

from zerollm import Chat

bot = Chat("Qwen/Qwen3-0.6B")

# Ask
print(bot.ask("Explain quantum computing in one sentence"))

# Stream
for token in bot.stream("Write a haiku about code"):
    print(token, end="", flush=True)

# Interactive REPL
bot.chat()

Agent with Tools

from zerollm import Agent

agent = Agent("Qwen/Qwen3-0.6B")

@agent.tool
def get_weather(city: str) -> str:
    """Get weather for a city."""
    return f"22°C and sunny in {city}"

print(agent.ask("What's the weather in Auckland?"))

Sub-Agents

researcher = Agent("Qwen/Qwen3-1.7B", name="researcher")

@researcher.tool
def search(query: str) -> str:
    return f"Results for: {query}"

main = Agent("Qwen/Qwen3-0.6B")
main.add_agent("researcher", researcher, "Research any topic")

main.ask("Research the latest AI trends")

Serve as API

from zerollm import Server

Server("Qwen/Qwen3-0.6B", port=8080).serve()

OpenAI-compatible. Works with any client that speaks the OpenAI API.

Fine-Tune

from zerollm import FineTuner

tuner = FineTuner("Qwen/Qwen3-0.6B")
tuner.train("my_data.csv", epochs=3)
tuner.save("my-bot")

Then serve your fine-tuned model:

from zerollm import Chat, Server

Chat("my-bot").ask("Hello!")        # chat with it
Server("my-bot", port=8080).serve() # or serve it

RAG

from zerollm import RAG

rag = RAG("Qwen/Qwen3-0.6B")
rag.add("docs.pdf")
print(rag.ask("What is the refund policy?"))

Powered by SQLite + sqlite-vec. No external database needed.

CLI

zerollm recommend                                          # best model for your hardware
zerollm chat Qwen/Qwen3-0.6B          # interactive chat
zerollm serve Qwen/Qwen3-0.6B         # start API server
zerollm list                                               # all available models
zerollm doctor                                             # diagnose setup

Supported Hardware

Platform	Acceleration	Auto-detected
Any CPU	llama.cpp	Yes
NVIDIA GPU	CUDA	Yes
Apple Silicon	Metal	Yes
AMD GPU	ROCm	Yes
Raspberry Pi	CPU	Yes

Models

Works with any GGUF model from Hugging Face. Pass the full HF model name or a local .gguf file:

Chat("Qwen/Qwen3-0.6B")  # from registry
Chat("/path/to/any-model.gguf")                # local file
Chat("my-finetuned-bot")                       # your fine-tuned model

Run zerollm list to see curated models, or zerollm recommend to find the best one for your hardware.

Architecture

License

MIT

Core Contributor

Nilesh Verma