freelm — free LLM API for Python & Node.js (OpenAI-compatible, auto-failover)

freelm turns the free tiers of Google Gemini, Groq, OpenRouter, Cloudflare Workers AI, Z.ai, Cohere, Mistral, NVIDIA NIM and Kilo into one OpenAI-compatible LLM — in your Python or TypeScript code, or as a local /v1 endpoint for any tool. A free-only, lightweight alternative to LiteLLM or an OpenRouter account: it rotates your keys, falls back across providers on rate limits, outages and retired models, races a slow provider against the next one (a hung provider costs ~6 s, not a 60 s failure — measured), and discovers which models are free today. A library, not a server to babysit: your own free keys go straight to each provider — no relay, dashboard or admin port, one runtime dependency in Python and none in Node — and the CLI even works with no keys at all.

pip install freelm          # Python >= 3.9   ·   npm install freelm  (Node >= 20, zero dependencies)

import freelm

llm = freelm.FreeLLM.from_env()      # reads GEMINI_API_KEY, GROQ_API_KEY, OPENROUTER_API_KEY, ...
print(llm.text("Explain failover in one sentence."))

Start in 60 seconds

Zero keys? pip install freelm && freelm chat "hello" already works: with no keys configured, the CLI falls back to keyless public endpoints (Kilo Gateway free routes, OVHcloud AI Endpoints anonymous tier) and says so. Their limits are low and free routes may log prompts, so for real use:

1. Get one free key (no credit card): Google AI Studio is the most generous; add Groq and OpenRouter for failover.

2. export GEMINI_API_KEY=... (and any others — see providers).

3. Check what works: freelm doctor sends one tiny request per key and tells you exactly what's wrong and how to fix it:

   $ freelm doctor
     openrouter  sk-or-...a1b2    FAIL  key rejected (401): User not found. — get a new free key: https://openrouter.ai/keys
     google      AIzaSy...x9yz    OK    gemini-3.5-flash-lite · 1131 ms
     groq        gsk_ab...cd12    FAIL  key rejected (401): Invalid API Key — get a new free key: https://console.groq.com/keys
   1 of 3 key(s) working — ready: google
   

4. Call it from code (above), the terminal (freelm chat "hi"), or any OpenAI-compatible tool (freelm serve, below).

Use free LLMs in any OpenAI-compatible tool

freelm serve runs the same router as a local OpenAI-compatible API — streaming, tool calls and failover included:

freelm serve                 # → http://127.0.0.1:4000/v1   (npx freelm serve works too)

Then point your tool at http://127.0.0.1:4000/v1 with any API key and model auto (or chat:fast, large, tools, or a concrete id):

Or with Docker (image built on each release):

docker run --rm -p 4000:4000 -e GEMINI_API_KEY=... -e FREELM_SERVER_KEY=change-me ghcr.io/shihabshahrier/freelm

GET /v1/models lists the aliases plus every discovered free model; GET /health shows key state. A model id no free provider offers (a tool's default gpt-4o) is tried, then served by auto (--no-fallback to disable). Bound to localhost by default — if you expose it (--host 0.0.0.0, Docker on Linux), set --api-key so nobody else spends your quota. In code: freelm.serve() (Python) / serve() (JS).

Python

from freelm import FreeLLM, GoogleAIStudio, Groq, OpenRouter

llm = FreeLLM(
    [GoogleAIStudio("AIza..."), Groq("gsk_..."), OpenRouter("sk-or-...")],   # or FreeLLM.from_env()
    strategy="smart",         # smart (default) | priority | round_robin | quota_aware | latency
)

r = llm.chat([{"role": "user", "content": "Write a haiku about failover."}], model="chat:fast")
print(r.text, "via", r.provider, r.model)

for chunk in llm.stream("Count to five."):          # token streaming, failover before the first token
    print(chunk, end="", flush=True)

r = llm.chat(msgs, model="chat:tools", tools=[...], tool_choice="auto")   # function calling
r.tool_calls

llm.chat(msgs, response_format={"type": "json_object"})                    # JSON mode

Async is symmetric (AsyncFreeLLM, await llm.chat(...), async for c in llm.astream(...)). For raw OpenAI chat.completion.chunk objects — tool-call deltas, finish reasons, usage — use stream_chunks() / astream_chunks().

Drop-in for the OpenAI SDK — change one import:

# from openai import OpenAI
from freelm.compat import OpenAI

client = OpenAI()   # api_key / base_url / ... are accepted and ignored; keys come from the environment
r = client.chat.completions.create(model="auto", messages=[{"role": "user", "content": "hi"}])
print(r.choices[0].message.content)

Responses behave like the SDK's (r.choices[0].message.tool_calls[0].function.arguments, model_dump(), stream=True chunks, with blocks, client.models.list()).

JavaScript / TypeScript

import { FreeLLM } from "freelm";

const llm = FreeLLM.fromEnv();
console.log(await llm.text("Explain failover in one sentence."));
for await (const chunk of llm.stream("Count to five.", { signal: AbortSignal.timeout(30_000) })) process.stdout.write(chunk);

Same engine, same providers, zero dependencies; ESM + CommonJS. No node: imports in the library, so it also bundles for edge functions, Workers and browsers (disk cache/persistence switch on only where Node built-ins exist). See the JS README.

Free providers

Beta = added in 0.5.0 and not yet verified with live keys — freelm doctor tells you if yours works. Several keys per provider: comma-separate them or use FREELM_<PROVIDER>_KEYS. The library never uses the keyless endpoints unless asked — FreeLLM.from_env(keyless=True) (always, as last resorts), keyless="auto" (only when no keys are set — the CLI's default) or FREELM_KEYLESS=1|auto|0. Any other OpenAI-compatible endpoint works too: Provider("key", name="myhost", base_url="https://.../v1", models=[ModelSpec("model-id", ("chat",))]).

Free tiers change constantly — models get retired, limits move, keys expire. freelm is built for that: it discovers models live (OpenRouter, Groq, Mistral, Kilo, OVHcloud, Cerebras), benches retired or overloaded models instead of retrying them, and freelm doctor tells you what works right now. The built-in fallback model lists were re-verified on 2026-10-09.

How it stays up

• A slow or hung provider doesn't hold the call. When an attempt is still running after a few seconds (3× that provider's usual latency, clamped; ~3 s for a stream's first token, ~6 s for a whole answer when it's unknown), the next provider starts in parallel and the first answer wins — the slow one is cancelled and remembered.
• Smart routing (default): providers are ranked by measured latency, so a fast one serves first and a slow one drops back; an unknown provider counts as typical so it gets tried, and old measurements expire after 10 minutes.
• Every failure fails over. 429 → rotate the key (or just bench the model, where quotas are per-model: Gemini, Groq, Z.ai, Cohere; a key whose daily or monthly quota is spent rests instead of being retried every minute); 5xx / timeouts → circuit breaker + backoff; 401/402 → disable that key; a retired model (404/410) → bench it for an hour; one provider rejecting a request (unsupported parameter, moderation) → try the next provider. Only a request that several providers reject is reported as your bug.
• Breadth-first failover: the best model of every provider is tried before any provider's second model, so a provider full of throttled or dead models can't stall the call.
• Quota guard: per-key requests/minute bucket + daily counter; keys predicted to be exhausted are skipped.
• Errors that explain themselves: when everything fails, NoProvidersAvailable says why per provider (openrouter: disabled (auth:401 — key invalid/expired?) | google: cooling down ~30s ...).

Choosing models

Bias selection with prefer=["gemini-2.5-flash", "gpt-oss"] (exact id or substring), order a static list with ModelSpec(priority=), or rank providers with priority= (lower = first).

Thinking models (gemini-2.5-flash, gpt-oss, ...) can spend a small max_tokens budget on hidden reasoning and return empty text — give them ≥ 128 tokens or use auto, which prefers non-thinking models.

Configuration

Provider options: tier, priority, prefer, models, rpm/rpd, discover, free_only (OpenRouter defaults to True: paid ids raise ConfigError instead of billing you), cache_ttl. llm.health() shows per-key state; llm.reset_keys() gives every key a fresh start.

CLI

freelm chat "explain failover in one line" --model chat:fast --stream
freelm doctor [--json]          # live-check every key, with fixes
freelm serve [--port 4000] [--api-key KEY] [--cors]
freelm models --provider groq   # live model list
freelm health                   # this process's key state

When can freelm cost money?

Free-only by default and by guard: OpenRouter, Kilo and Z.ai paid models are blocked unless you pass free_only=False; Google is free unless you pick tier="tier1" (billing enabled); NIM consumes free build credits and Cerebras trial credits (requests fail when they run out — nothing is billed through freelm); Groq and Mistral free accounts only have free models; OVHcloud is used anonymously only. Your account's plan decides the rest: Cohere bills only production (non-trial) keys, and Cloudflare only on the Workers Paid plan (past the daily free Neurons — the Free plan just stops).

How freelm compares

Most free-LLM tools of 2026 are servers you run — many with a dashboard, most aimed at coding agents. freelm is the library for your own code, with the server optional:

FAQ

What is the best free LLM API in 2026?

Google AI Studio (Gemini) has the most generous free tier; Groq is the fastest; OpenRouter has the widest choice of :free models. Each has its own limits and outages, so the most reliable option is to use several at once — freelm pools them behind one OpenAI-compatible call and fails over automatically.

Is there a free alternative to the OpenAI API?

Yes. freelm.compat.OpenAI is a drop-in for the OpenAI SDK, and freelm serve exposes a local /v1/chat/completions endpoint, both backed by free tiers (Gemini, Groq, OpenRouter, Cloudflare Workers AI, Z.ai, Cohere, Mistral, NVIDIA NIM) with streaming, tool calling and automatic failover.

How do I use free LLMs in OpenCode, OpenClaw, Hermes, Cline, Continue, Open WebUI or n8n?

Run freelm serve and set the tool's OpenAI base URL to http://127.0.0.1:4000/v1 with model auto — see the table above. Tools that call the endpoint from their own servers need a public URL (for example a tunnel) plus --api-key.

How do I use free LLMs in Python or JavaScript?

pip install freelm or npm install freelm, set one free key (e.g. GEMINI_API_KEY), then freelm.FreeLLM.from_env().text("...") / await FreeLLM.fromEnv().text("...").

Can I use an LLM API with no API key at all?

Yes, for trying things out: freelm chat "hello" (or npx freelm chat "hello") works with no keys by using documented keyless endpoints — Kilo Gateway's free routes (~200 requests/hour per IP) and OVHcloud AI Endpoints' anonymous tier. Limits are low and free routes may log prompts; one free Gemini or Groq key is far more capable.

Why did my free model stop working?

Free model ids are retired constantly (NVIDIA retired its Llama 3.x endpoints on 2026-08-26; Groq retired Llama 3.3 70B on 2026-08-16). freelm discovers current models live, benches retired ones on the first 404/410 and fails over, so your code keeps working; freelm doctor shows which keys and models work today.

How fast is failover?

An error (429, 5xx, bad key, retired model) fails over immediately — the cost is one extra round trip, usually well under a second. A provider that hangs or answers slowly is raced: after a few seconds the next provider starts in parallel and the first answer wins (measured: a hung provider ahead of a healthy one answers in ~6 s, a stalled stream in ~3 s). After that, smart routing sends calls to the fast provider first, so later calls don't wait at all. The numbers come from a reproducible simulation: benchmarks/. A host that won't accept a connection fails within 10 s.

Is there a lightweight LiteLLM alternative?

For free tiers, yes: freelm is one dependency in Python (httpx) and zero in Node, with no proxy to run. LiteLLM is the better tool for paid production traffic across 100+ providers, budgets and teams; freelm is narrower — free-only, with per-key quota and model state, free-model discovery and failover tuned to how free tiers break.

Is there a free OpenRouter alternative?

Use the free tiers at the source: Google AI Studio, Groq, Cloudflare Workers AI, Mistral, NVIDIA NIM, Z.ai and Cohere all serve real models at no cost. freelm calls them directly with your own keys and keeps OpenRouter as one more pool, so when OpenRouter's free models are throttled the call moves on instead of failing.

How do I add LLM fallback or failover in Python or Node.js?

FreeLLM.from_env() (or FreeLLM.fromEnv()) already falls back across every provider you have a key for: errors move on in one round trip, slow providers are raced, and model=["vendor/id", "chat:fast"] gives an explicit per-call fallback chain. No try/except needed.

How do I rotate API keys across free LLM tiers?

Comma-separate several keys per provider (GROQ_API_KEY=key1,key2) or pass a list: freelm paces each key, rotates on 429, cools or disables keys individually and, with persist=True, remembers quota state across restarts.

What does this free-tier error mean?

Each common one has a guide with the exact error body, the causes, the fix on the provider itself and what freelm does about it: Gemini 429 RESOURCE_EXHAUSTED, Gemini "User location is not supported", Gemini "API key not valid", Gemini 503 overloaded, OpenRouter free-models-per-day, OpenRouter "No endpoints found that support tool use", Groq rate limits, Mistral 429, Z.ai 1302 and empty replies from thinking models — all in the error reference.

How do I avoid free-tier rate limits (429)?

Add more providers and keys: freelm paces each key, rotates on 429, benches per-model quotas (Gemini, Groq) and spreads load with strategy="quota_aware".

Is freelm really free? Does it see my prompts?

freelm is MIT-licensed and runs in your process (or on your machine with freelm serve); requests go straight from you to the providers whose keys you configure. It has no telemetry. Usage limits are those of each provider's free tier.

Links

Changelog · Contributing · Security · Website & docs · PyPI · npm

MIT © Shahriar Labs