freelm — free LLM API for Python & Node.js (OpenAI-compatible, auto-failover)

freelm turns the free tiers of Google Gemini, Groq, OpenRouter, Mistral, NVIDIA NIM and Kilo into one OpenAI-compatible LLM — in your Python or TypeScript code, or as a local /v1 endpoint for any tool. It rotates your keys, fails over across providers on rate limits, outages and retired models, and discovers which models are free today. Your own free keys, called directly: nothing to host, no relay in the middle — and the CLI even works with no keys at all.

pip install freelm          # Python >= 3.9   ·   npm install freelm  (Node >= 20, zero dependencies)

import freelm

llm = freelm.FreeLLM.from_env()      # reads GEMINI_API_KEY, GROQ_API_KEY, OPENROUTER_API_KEY, ...
print(llm.text("Explain failover in one sentence."))

Start in 60 seconds

Zero keys? pip install freelm && freelm chat "hello" already works: with no keys configured, the CLI falls back to keyless public endpoints (Kilo Gateway free routes, OVHcloud AI Endpoints anonymous tier) and says so. Their limits are low and free routes may log prompts, so for real use:

1. Get one free key (no credit card): Google AI Studio is the most generous; add Groq and OpenRouter for failover.

2. export GEMINI_API_KEY=... (and any others — see providers).

3. Check what works: freelm doctor sends one tiny request per key and tells you exactly what's wrong and how to fix it:

   $ freelm doctor
     openrouter  sk-or-...a1b2    FAIL  key rejected (401): User not found. — get a new free key: https://openrouter.ai/keys
     google      AIzaSy...x9yz    OK    gemini-2.5-flash-lite · 1131 ms
     groq        gsk_ab...cd12    FAIL  key rejected (401): Invalid API Key — get a new free key: https://console.groq.com/keys
   1 of 3 key(s) working — ready: google
   

4. Call it from code (above), the terminal (freelm chat "hi"), or any OpenAI-compatible tool (freelm serve, below).

Use free LLMs in any OpenAI-compatible tool

freelm serve runs the same router as a local OpenAI-compatible API — streaming, tool calls and failover included:

freelm serve                 # → http://127.0.0.1:4000/v1   (npx freelm serve works too)

Then point your tool at http://127.0.0.1:4000/v1 with any API key and model auto (or chat:fast, large, tools, or a concrete id):

Or with Docker (image built on each release):

docker run --rm -p 4000:4000 -e GEMINI_API_KEY=... -e FREELM_SERVER_KEY=change-me ghcr.io/shihabshahrier/freelm

GET /v1/models lists the aliases plus every discovered free model; GET /health shows key state. A model id no free provider offers (a tool's default gpt-4o) is tried, then served by auto (--no-fallback to disable). Bound to localhost by default — if you expose it (--host 0.0.0.0, Docker on Linux), set --api-key so nobody else spends your quota. In code: freelm.serve() (Python) / serve() (JS).

Python

from freelm import FreeLLM, GoogleAIStudio, Groq, OpenRouter

llm = FreeLLM(
    [GoogleAIStudio("AIza..."), Groq("gsk_..."), OpenRouter("sk-or-...")],   # or FreeLLM.from_env()
    strategy="quota_aware",   # priority | round_robin | quota_aware | latency
)

r = llm.chat([{"role": "user", "content": "Write a haiku about failover."}], model="chat:fast")
print(r.text, "via", r.provider, r.model)

for chunk in llm.stream("Count to five."):          # token streaming, failover before the first token
    print(chunk, end="", flush=True)

r = llm.chat(msgs, model="chat:tools", tools=[...], tool_choice="auto")   # function calling
r.tool_calls

llm.chat(msgs, response_format={"type": "json_object"})                    # JSON mode

Async is symmetric (AsyncFreeLLM, await llm.chat(...), async for c in llm.astream(...)). For raw OpenAI chat.completion.chunk objects — tool-call deltas, finish reasons, usage — use stream_chunks() / astream_chunks().

Drop-in for the OpenAI SDK — change one import:

# from openai import OpenAI
from freelm.compat import OpenAI

client = OpenAI()   # api_key / base_url / ... are accepted and ignored; keys come from the environment
r = client.chat.completions.create(model="auto", messages=[{"role": "user", "content": "hi"}])
print(r.choices[0].message.content)

Responses behave like the SDK's (r.choices[0].message.tool_calls[0].function.arguments, model_dump(), stream=True chunks, with blocks, client.models.list()).

JavaScript / TypeScript

import { FreeLLM } from "freelm";

const llm = FreeLLM.fromEnv();
console.log(await llm.text("Explain failover in one sentence."));
for await (const chunk of llm.stream("Count to five.", { signal: AbortSignal.timeout(30_000) })) process.stdout.write(chunk);

Same engine, same providers, zero dependencies; ESM + CommonJS. No node: imports in the library, so it also bundles for edge functions, Workers and browsers (disk cache/persistence switch on only where Node built-ins exist). See the JS README.

Free providers

Several keys per provider: comma-separate them or use FREELM_<PROVIDER>_KEYS. The library never uses the keyless endpoints unless asked — FreeLLM.from_env(keyless=True) (always, as last resorts), keyless="auto" (only when no keys are set — the CLI's default) or FREELM_KEYLESS=1|auto|0. Any other OpenAI-compatible endpoint works too: Provider("key", name="myhost", base_url="https://.../v1", models=[ModelSpec("model-id", ("chat",))]).

Free tiers change constantly — models get retired, limits move, keys expire. freelm is built for that: it discovers models live (OpenRouter, Groq, Mistral, Kilo, OVHcloud, Cerebras), benches retired or overloaded models instead of retrying them, and freelm doctor tells you what works right now. The built-in fallback model lists were re-verified on 2026-10-09.

How it stays up

• Every failure fails over. 429 → rotate the key (or just bench the model, where quotas are per-model: Gemini, Groq); 5xx / timeouts → circuit breaker + backoff; 401/402 → disable that key; a retired model (404/410) → bench it for an hour; one provider rejecting a request (unsupported parameter, moderation) → try the next provider. Only a request that several providers reject is reported as your bug.
• Breadth-first failover: the best model of every provider is tried before any provider's second model, so a provider full of throttled or dead models can't stall the call.
• Quota guard: per-key requests/minute bucket + daily counter; keys predicted to be exhausted are skipped.
• Errors that explain themselves: when everything fails, NoProvidersAvailable says why per provider (openrouter: disabled (auth:401 — key invalid/expired?) | google: cooling down ~30s ...).

Choosing models

Bias selection with prefer=["gemini-2.5-flash", "gpt-oss"] (exact id or substring), order a static list with ModelSpec(priority=), or rank providers with priority= (lower = first).

Thinking models (gemini-2.5-flash, gpt-oss, ...) can spend a small max_tokens budget on hidden reasoning and return empty text — give them ≥ 128 tokens or use auto, which prefers non-thinking models.

Configuration

Provider options: tier, priority, prefer, models, rpm/rpd, discover, free_only (OpenRouter defaults to True: paid ids raise ConfigError instead of billing you), cache_ttl. llm.health() shows per-key state; llm.reset_keys() gives every key a fresh start.

CLI

freelm chat "explain failover in one line" --model chat:fast --stream
freelm doctor [--json]          # live-check every key, with fixes
freelm serve [--port 4000] [--api-key KEY] [--cors]
freelm models --provider groq   # live model list
freelm health                   # this process's key state

When can freelm cost money?

Free-only by default and by guard: OpenRouter and Kilo paid models are blocked unless you pass free_only=False; Google is free unless you pick tier="tier1" (billing enabled); NIM consumes free build credits and Cerebras trial credits (requests fail when they run out — nothing is billed through freelm); Groq and Mistral free accounts only have free models; OVHcloud is used anonymously only.

How freelm compares

FAQ

What is the best free LLM API in 2026?

Google AI Studio (Gemini) has the most generous free tier; Groq is the fastest; OpenRouter has the widest choice of :free models. Each has its own limits and outages, so the most reliable option is to use several at once — freelm pools them behind one OpenAI-compatible call and fails over automatically.

Is there a free alternative to the OpenAI API?

Yes. freelm.compat.OpenAI is a drop-in for the OpenAI SDK, and freelm serve exposes a local /v1/chat/completions endpoint, both backed by free tiers (Gemini, Groq, OpenRouter, Cerebras, Mistral, NVIDIA NIM) with streaming, tool calling and automatic failover.

How do I use free LLMs in Cursor, Cline, Continue, Open WebUI or n8n?

Run freelm serve and set the tool's OpenAI base URL to http://127.0.0.1:4000/v1 with model auto — see the table above. Tools that call the endpoint from their own servers need a public URL (for example a tunnel) plus --api-key.

How do I use free LLMs in Python or JavaScript?

pip install freelm or npm install freelm, set one free key (e.g. GEMINI_API_KEY), then freelm.FreeLLM.from_env().text("...") / await FreeLLM.fromEnv().text("...").

Can I use an LLM API with no API key at all?

Yes, for trying things out: freelm chat "hello" (or npx freelm chat "hello") works with no keys by using documented keyless endpoints — Kilo Gateway's free routes (~200 requests/hour per IP) and OVHcloud AI Endpoints' anonymous tier. Limits are low and free routes may log prompts; one free Gemini or Groq key is far more capable.

Why did my free model stop working?

Free model ids are retired constantly (NVIDIA retired its Llama 3.x endpoints on 2026-08-26; Groq retired Llama 3.3 70B on 2026-08-16). freelm discovers current models live, benches retired ones on the first 404/410 and fails over, so your code keeps working; freelm doctor shows which keys and models work today.

How do I avoid free-tier rate limits (429)?

Add more providers and keys: freelm paces each key, rotates on 429, benches per-model quotas (Gemini, Groq) and spreads load with strategy="quota_aware".

Is freelm really free? Does it see my prompts?

freelm is MIT-licensed and runs in your process (or on your machine with freelm serve); requests go straight from you to the providers whose keys you configure. It has no telemetry. Usage limits are those of each provider's free tier.

Links

Changelog · Contributing · Security · Website & docs · PyPI · npm

MIT © Shahriar Labs