kllm runs open-weight LLMs on Kaggle's free GPUs and gives your coding tools a local API for them.
Your laptop has no GPU. You still want Claude Code to refactor a 2,000-line module on a 35B model, with a
256K context window, without paying per token. With kllm you install one package, paste a Kaggle token once,
and run claude-kllm. Ten minutes later the model is answering from two Tesla T4s in a Kaggle notebook, and
Claude Code talks to it as if it were running on 127.0.0.1.
That raises a few questions, answered below:
- How do I get started?
- What happens when I run it?
- Which models can I run, and how fast are they?
- How do I connect Claude Code, opencode or my own code?
- What happens when Kaggle's 12-hour limit hits?
- Is my code and my Kaggle token safe?
How do I get started?
1. Install with one command. The one-line installers bring their own Python through uv.
| You have | Run |
|---|---|
| Nothing (Linux, macOS) | curl -fsSL https://raw.githubusercontent.com/SANTHAN-KUMAR/kllm/main/install.sh | sh |
| Nothing (Windows) | powershell -ExecutionPolicy ByPass -c "irm https://raw.githubusercontent.com/SANTHAN-KUMAR/kllm/main/install.ps1 | iex" |
| uv | uv tool install kllm-gateway |
| pipx | pipx install kllm-gateway |
| pip | pip install kllm-gateway |
The package is kllm-gateway on PyPI. It installs two commands: kllm and claude-kllm.
2. Run kllm. It starts the local relay and opens the control panel in your browser. On a first run the
panel shows a four-step setup: add your Kaggle account, pick a model, connect a tool, say hello.
First run: the panel links to your Kaggle settings and takes the token as a paste or a dropped kaggle.json. No file to edit.
You need a Kaggle account with a verified phone number, because Kaggle only gives GPUs to verified accounts.
If ~/.kaggle/kaggle.json already exists, kllm finds it and skips the paste.
3. Use it.
claude-kllm # Claude Code on the default model
opencode -m kllm/qwen3.6-35b-abliterated # opencode
kllm down # stop the GPU when you are done, to save quota
The first request starts a GPU if none is running. Expect about 8 to 10 minutes for a cold start, while Kaggle boots the notebook and downloads the model. The request waits and then continues on its own.
What happens when I run it?
Your tools never talk to Kaggle. They talk to a small relay on your PC (one Python process, about 55 MB of RAM), and the relay does the Kaggle work.
sequenceDiagram
autonumber
participant T as Claude Code
participant R as kllm relay (your PC)
participant K as Kaggle notebook (2x T4)
T->>R: first request for qwen3.6-35b-abliterated
R->>K: push a private notebook with your Kaggle token
Note over K: build llama.cpp, download the model, open a tunnel
K-->>R: tunnel URL, sealed and signed
R->>K: forward the request through the tunnel
K-->>T: streamed answer (the relay sends keep-alives while it waits)
The relay holds the first request while the GPU starts, so the tool needs no retry logic.
After that, every request goes straight from the relay through the tunnel. The relay keeps each conversation on the same GPU so the prompt cache stays warm: in measured sessions, 99% of each turn's prompt came from the cache at 55K tokens of context. Details are in How kllm works.
Which models can I run, and how fast are they?
Two models are measured end to end on two T4s. The panel lists 46 models in all, and adds any GGUF or diffusers repo from Hugging Face.
| Model | Best for | Context | Measured speed |
|---|---|---|---|
qwen3.6-35b-abliterated (default) |
Long agent sessions; thinks before answering | 256K | 70 tok/s short, 43 tok/s at 209K |
qwen3-coder-30b-abliterated |
Fastest turnaround, no thinking | 96K | 84 tok/s short, 35 tok/s at 84K |
The model library shows what fits two T4s before you launch anything.
Image and video models (SDXL Turbo, LTX-Video, Wan 2.1) run in the same notebook behind the OpenAI Images API. Anything above 30 GB of VRAM does not fit. See Models for adding models and calling image and video endpoints.
How do I connect Claude Code, opencode or my own code?
The relay speaks both the OpenAI and the Anthropic API at http://127.0.0.1:8080. Requests from your own
machine need no key.
| Tool | How |
|---|---|
| Claude Code | claude-kllm, which sets the model and the real context window for you |
| opencode, Codex CLI, Gemini CLI, Antigravity, Cursor | the panel's Connect page writes their config and keeps a backup for Undo |
| Any OpenAI client | base URL http://127.0.0.1:8080/v1, model kllm (whatever is running) |
| Any Anthropic client | ANTHROPIC_BASE_URL=http://127.0.0.1:8080 |
From your own code it is a normal chat call:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="local")
reply = client.chat.completions.create(
model="kllm",
messages=[{"role": "user", "content": "Write a SQL query for the 5 newest orders."}],
)
print(reply.choices[0].message.content)
Other devices on your network, or a public URL, can use the same API with the gateway key shown in the panel's Settings. See Configuration.
What happens when Kaggle's 12-hour limit hits?
A Kaggle notebook stops after 12 hours. kllm plans for it: 25 minutes before the limit it starts a successor, replays each conversation's latest prompt there so the cache is warm, and only then moves the conversation.
gantt
title One long Claude Code session across two Kaggle notebooks
dateFormat HH:mm
axisFormat %H:%M
section Notebook A
serving requests :a1, 00:00, 11h
section Notebook B
starting (8 to 10 min) :b0, 10:35, 10m
warm handoff :crit, b1, 10:45, 5m
serving requests :b2, 10:50, 6h
The successor is ready before the old notebook stops, so the session never sees the 12-hour cap.
The same handoff runs when a notebook dies early. In a 72-minute test with three agent sessions in parallel, contexts grew to 165K tokens and generation held at 50 to 63 tok/s from empty to 175K (full results).
Each Kaggle account gets about 30 GPU hours a week. kllm down stops the GPU, and an idle GPU stops by itself
after 25 minutes without requests.
Is my code and my Kaggle token safe?
- Your Kaggle token stays on your PC, in your user folder. kllm uses it only to call Kaggle's API from your machine; it is never put into the notebook or sent anywhere else.
- Your prompts go only to your notebook, through a tunnel protected by a per-notebook API key.
- The notebook's address is sealed. The notebook reports its tunnel URL through ntfy.sh, encrypted and signed with a secret generated on your PC, so nobody can read it or swap in a fake URL.
- The panel is local only. Requests from other machines need the gateway key and can reach only the model API.
kllm uses your Kaggle account and its free quota. Follow Kaggle's Terms of Service, which allow one account per person. kllm can manage several accounts (for example a team where each person adds their own), but which accounts you add is your responsibility.
What does not work yet?
- The first launch of a model on an account downloads about 20 GB, so it takes 8 to 10 minutes. Kaggle's GPU queue occasionally adds more.
- Models above about 28 GB of weights plus cache do not fit two T4s. A 230B model such as MiniMax-M2 cannot run.
- The bundled image and video entries are marked unverified: they were sized and checked, but not run on a GPU.
- Qwen3.6 sometimes thinks at length on trivial requests. That costs seconds, not quality.
Back to that refactor
Claude Code is working through the 2,000-line module. It started on Notebook A at 09:00, the relay moved it to Notebook B at 19:45 with its cache warm, and it is still answering at 43 tok/s with 200K tokens of context. Nothing ran on your laptop but a 55 MB relay.
More
| Read | For |
|---|---|
| How kllm works | speed tuning, loop guard, tunnel keep-alive, GPU scheduling, the long-session test |
| Models | choosing a model, adding one from Hugging Face, image and video endpoints |
| Configuration | every tool's setup, remote access, gateway key, file layout |
| Contributing | running the tests |
Contributing
Issues and pull requests are welcome. Run the tests with pip install -e ".[dev]" and python -m pytest.
They never contact Kaggle, GitHub or ntfy, and never touch your home directory.
© 2026 SANTHAN-KUMAR. Released under the MIT license.
Metadata
Release files for kllm-gateway 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| kllm_gateway-0.3.0.tar.gz | 4.3 MB | With provenance | Details for kllm_gateway-0.3.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| kllm_gateway-0.3.0-py3-none-any.whl | Python 3 | none | any | Details for kllm_gateway-0.3.0-py3-none-any.whl |
Total release size: 8.0 MB
Release files / kllm_gateway-0.3.0.tar.gz
| Download URL | kllm_gateway-0.3.0.tar.gz |
|---|---|
| Size | 4.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7e37bd5d6a35eb6a32554503c25de8aba3cd8d0a583fb6b1f2078cc5af665096
|
|
BLAKE2b-256 checksum How to use checksums |
88e216979f74931bc70f4d68c2702e1f6bb122ef558688898cd9fbd103174f57
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logRelease files / kllm_gateway-0.3.0-py3-none-any.whl
| Download URL | kllm_gateway-0.3.0-py3-none-any.whl |
|---|---|
| Size | 3.7 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9fd134fa7fb3f18a1ac9326807eb9458c479f3f10f71f1ab3c6d45cf74223e42
|
|
BLAKE2b-256 checksum How to use checksums |
29d49af69eb85170e7650555c766e6694d2d94c7da97331be871d14d51a31f22
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logSecurity for kllm-gateway 0.3.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for kllm_gateway-0.3.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.