freemodel-sync

English | 简体中文

Discover, test and rank free LLMs, then write the top-ranked working one (plus fallbacks) into the configs of tools you already use.

Free LLM endpoints come and go: models get added, rate-limited, deprecated or quietly start costing money. freemodel-sync points your bots and scripts at a free model that passed a recent probe. It cannot guarantee the model keeps working between runs.

Not a proxy. freemodel-sync never sees or forwards your traffic. It only edits config files (with a backup) and then gets out of the way. Your tools keep talking to the provider directly.

How it works

discover ──► filter ──► probe (tiny, cached, budgeted) ──► rank ──► select ──► write configs ──► notify

1. Discover free models from the public OpenRouter catalogue (GET /api/v1/models, no key needed). A model counts as free only if its id ends in :free, prompt and completion prices are 0, and it takes and returns text. The openrouter/free meta-router is excluded.
2. Filter by tool-calling support, minimum context window, id globs, and expiry: models whose expiration_date is within N days are dropped (expired models are always dropped, even with exclude_expiring_within_days: null), and models expiring within 30 days are penalised.
3. Probe each candidate with one tiny streaming request (max_tokens: 16 by default, providers.openrouter.probe_max_tokens) using your own OpenRouter key, measuring availability and time-to-first-token. Probing is frugal: candidates are tried in order of promise, results are cached (6 h by default), at most max_tests new probes are made per run (at least 1), and the round stops while keeping reserve_requests of the free requests left today, as reported by free_model_daily_requests.remaining in GET /api/v1/key (UTC day; per OpenRouter's documentation at the time of writing, 50/day below 10 purchased credits and 1000/day above). If the key endpoint reports nothing, a conservative 50/day is assumed against a local UTC-day counter.
   • Account problems are not model failures. A 401, a 402, an OpenRouter platform 429 (free-model per-minute/per-day cap) or a 403 about the account (key permissions, guardrails, budget limits, or a bare 403 with no details) stops the round after that single request; the model is not cached as failed, and Retry-After / X-RateLimit-Reset pause later runs until the reset. A 403 whose error metadata points at the model itself (routing or moderation details, e.g. a free endpoint that refuses this kind of client, or flagged input) is a failure of that model only: it is cached as failed and probing goes on with the next model. Upstream-provider 429s, 408, 5xx and timeouts count as transient failures and are retried after transient_ttl_minutes (30 by default). So do errors that arrive inside the stream after HTTP 200, such as a mid-stream rate limit ("temporarily rate-limited upstream"): probing goes on with the next model; only an explicit 401/402 or account-level 403 in the stream still stops the round. Other errors (e.g. 400/404 "no endpoints") are model failures. On Groq, every 401, 403 and 429 is treated as an account problem.
   • Without an API key, test, rank and sync exit with an error and send nothing.
4. Rank with an explainable score (availability ≫ latency, context, tools, expiry). Fallbacks are models that passed their probe; untested models are used only if you set ranking.fallbacks_allow_untested: true, and are then marked (untested). Non-chat models (safety classifiers, embedding, rerank, speech) are excluded by default.
5. Write the primary model and fallback list into each configured target. Every write is idempotent (unchanged → untouched), supports --dry-run with a diff, makes a timestamped *.bak-* backup (mode 0600 or stricter, the newest 5 kept by default; see backups) and replaces the file atomically. If no model passes its probe, nothing is written.
6. Notify (optional) via Telegram or a generic webhook when the primary model changes. If any target fails to write, the run exits 1, the recorded state is not advanced and no notification is sent; the next run retries.

Install

pipx install freemodel-sync              # latest release from PyPI
pipx install git+https://github.com/ndwlbj/freemodel-sync   # unreleased changes from main
# or run without installing
uvx --from git+https://github.com/ndwlbj/freemodel-sync freemodel-sync --help
# from a local checkout
pipx install .

Requires Python 3.9+ (Python 3.9 is end-of-life; a future release is planned to require 3.10+). Runtime dependencies: httpx, PyYAML.

Quick start

export OPENROUTER_API_KEY=...            # your own key; only read from the environment
freemodel-sync discover                  # list free models (no key needed)
freemodel-sync test --max-tests 3        # probe the 3 most promising ones
freemodel-sync rank --json               # full ranking + selection as JSON
cp examples/config.example.yml ~/.config/freemodel-sync/config.yml   # then uncomment and edit targets
freemodel-sync sync --dry-run            # show the diffs
freemodel-sync sync                      # write configs, notify on change

Run sync from cron or a systemd timer, e.g. every 3 hours. Keep the key out of the crontab: put export OPENROUTER_API_KEY=... in a file only you can read (chmod 600 ~/.config/freemodel-sync/env) and source it, or use EnvironmentFile= in a systemd unit.

0 */3 * * * . "$HOME/.config/freemodel-sync/env" && freemodel-sync sync >> "$HOME/.cache/freemodel-sync/sync.log" 2>&1

Exit codes: 0 success, 1 error (bad config, missing key, a target failed), 2 no usable model (no file written), 3 probing was stopped by an account/quota problem such as a platform rate limit or missing credits (no file written, no model marked as failed), 4 another run is still in progress (test, rank and sync take a lock next to the state file, so overlapping cron runs never probe twice). test and rank both return 3 when an account/quota problem stopped probing.

sync --dry-run is zero-cost: it uses only cached probe results (from an earlier test/sync), sends no chat or /key request, needs no API key, and writes no target, state or cache file. It still fetches the public model list. Add --probe to probe as usual during a dry run (this spends free-tier requests and updates the cache). rank --no-probe and sync --no-probe likewise use cached results only.

Configuration

Default location: ~/.config/freemodel-sync/config.yml (override with -c FILE or $FREEMODEL_SYNC_CONFIG). Every key is optional; see examples/config.example.yml for the full annotated file. Its targets and notification channels are commented out: uncomment the ones you use and point them at your own files.

• An unknown key (usually a typo such as testing.max_test) is ignored with a warning that names it. Keys under providers.<name> are provider options and are not checked.
• On/off options (notify.*.enabled, the dsh option check_model, preserve_comments) accept true/false, yes/no, on/off and 1/0, quoted or not. Any other value is an error rather than a guess.
• A target file that does not exist is created with only the managed settings, and a warning is logged (often the path is a typo or a placeholder).
• With no targets, sync writes no file but still records the selection in the state file and sends notifications when it changes (a notify-only setup).

Providers

Groq and multi-provider ranking need freemodel-sync 0.3.0 or newer. All enabled providers feed one ranking. The providers section replaces the default (OpenRouter only), so you can drop OpenRouter or set enabled: false. Each provider uses its own key variable, cache file, daily counter and rate-limit pause, and a quota problem stops probing only for that provider. Fallbacks always come from the primary's provider. Restrict a target to the providers its tool has a key for with options.providers: [openrouter]; otherwise its base URL may switch provider. Details and sources are in docs/providers.md.

providers:
  openrouter: {}
  groq: {plan: free}          # only if your Groq org is on the Free plan
targets:
  - adapter: openai-env
    path: ~/bot/.env
    options: {providers: [openrouter]}   # this bot only has an OpenRouter key

Targets (adapters)

Example dsh result:

- id: agent-default-model
  config: {provider: openrouter, model: 'vendor/some-model:free'}

Known limitations

• AstrBot: both layouts were checked against AstrBot source (v4.29.1); an entry matching neither raises an error instead of being skipped. AstrBot keeps its config in memory and rewrites the file when you save in the WebUI, so restart AstrBot after a sync (and avoid editing in the WebUI at the same time). Provider entries have no fallback list, so only the primary model is written; mapping fallbacks to AstrBot's fallback_provider_ids is not implemented yet.
• YAML comments (0.3.0+): install the optional extra (pipx install "freemodel-sync[yaml-comments]") and openai-yaml edits the file with ruamel.yaml, keeping comments, key order and quoting. Without it, PyYAML re-dumps the whole document when a value changes (an unchanged file is never rewritten): comments are dropped and unrelated values can be re-typed or re-quoted (1.10 becomes 1.1, yes becomes true, 0755 becomes 493, "yes" becomes 'yes'). So with the default preserve_comments: auto and no ruamel.yaml, a file that would lose anything is refused (the target fails and the file is left unchanged); a file PyYAML writes back as is (for example one freemodel-sync already wrote this way) is still updated. preserve_comments: true requires ruamel.yaml; false always uses PyYAML and accepts the re-dump. The dsh adapter always preserves comments.
• Time-based pricing: a :free model whose pricing.overrides has a non-zero price (for example at peak hours or for long contexts) is skipped with a warning.
• Free quota: the daily budget comes from free_model_daily_requests (never inferred from is_free_tier); testing.daily_free_limit overrides it, counted per UTC day. The counters are read once, before probing, and OpenRouter's used count can trail requests that were already sent, so shown quota figures may lag. When this tool's own probe count for the UTC day is higher than the reported used, the budget uses the local count.
• Providers: OpenRouter and Groq. SiliconFlow was evaluated but its API does not say which models are free; see docs/providers.md.

Notifications

• Telegram: set notify.telegram and export TELEGRAM_BOT_TOKEN / TELEGRAM_CHAT_ID.
• Webhook (e.g. a QQ bot bridge): set notify.webhook.url_env (default FREEMODEL_SYNC_WEBHOOK_URL). Webhook URLs usually contain a token, so keep them in the environment: an inline notify.webhook.url works but logs a warning, plus another if the config file is readable by other users. The POST body is {"event": "primary_changed", "old", "new", "fallbacks", "text"}.

Notification failures are logged (without tokens or full URLs) and never fail a sync. A configured channel whose environment variables are not set logs a warning when a notification is due, so a misspelt variable name does not go unnoticed; remove the channel (or set enabled: false) if you do not use it.

Daily health check

The repository runs an automated daily health check of free models with GitHub Actions (daily-snapshot.yml).

• When: scheduled with cron 17 1 * * *, i.e. 01:17 UTC, about 09:17 Beijing time (UTC+8). GitHub can start scheduled runs late, or skip them when it is busy; the workflow can also be started by hand.
• What: it discovers the free models of OpenRouter (and of Groq, only when a Groq key is configured and the Groq organization is confirmed to be on the Free plan). It then probes at most 15 models per provider, best static score first, with one tiny request each, staying within the free-request quota and keeping 10 requests in reserve. An account or quota error stops probing for that provider.
• Where: results are committed to the public data branch (main is never touched): latest snapshot, plus latest.json and one snapshots/YYYY-MM-DD.json per day in the snapshot folder.

The table shows each model's status (ok, 403, 404, 429, timeout, ...), latency and context length, plus the changes since the previous snapshot and models expiring soon. It is informational only: each result is one request from GitHub Actions at one moment and can differ for you, so run freemodel-sync test with your own key before relying on a model.

Writing a provider

Providers are plugins. Subclass Provider, implement discover() and test() (and optionally quota()), and register it:

from freemodel_sync.providers import Provider, register_provider

@register_provider
class MyProvider(Provider):
    name = "myprovider"
    base_url = "https://api.example.com/v1"
    api_key_env = "MYPROVIDER_API_KEY"

    def discover(self): ...
    def test(self, model, timeout=30.0): ...

Possible future providers (not implemented, no timeline): Zhipu GLM Flash, Google Gemini. Adapters follow the same pattern (freemodel_sync.adapters.base.Adapter + register_adapter).

Development

python -m pip install -e ".[dev]"
ruff check src tests scripts
ruff format --check src tests scripts
pytest

Tests never touch the network (HTTP is mocked with httpx.MockTransport). See CONTRIBUTING.md.

Security

To report a vulnerability privately, see SECURITY.md.

• Keys are read only from environment variables; freemodel-sync does not write them to config files or print them. Log output (including -v) redacts Telegram bot tokens, Bearer tokens and sk- style keys, and HTTP request logging (--debug-http) is off by default.
• Symlinked targets are followed: the real file is backed up and replaced, the link is kept.
• Never commit your config.yml, .env files, backups or cache; the .gitignore covers the defaults.

License

MIT © freemodel-sync contributors