freemodel-sync

English | 简体中文

Discover, test and rank free LLMs, then write the top-ranked working one (plus fallbacks) into the configs of tools you already use.

Free LLM endpoints come and go: models get added, rate-limited, deprecated or quietly start costing money. freemodel-sync points your bots and scripts at a free model that passed a recent probe. It cannot guarantee the model keeps working between runs.

Not a proxy. freemodel-sync never sees or forwards your traffic. It only edits config files (with a backup) and then gets out of the way. Your tools keep talking to the provider directly.

Two ways to use it

• Python CLI (pip install freemodel-sync): discovers, probes and ranks free models, then writes the winner into your tools' config files. Runs anywhere, e.g. from cron or CI.
• dsh plugin (dsh-freemodel, in plugins/dsh-freemodel/): a self-contained TypeScript plugin for dsh. It lists the free OpenRouter, Groq, SenseNova and Bailian models that passed a probe directly in the dsh model picker, with capability and quota badges, a settings card, a /freemodel command and dsh-freemodel status / doctor. Keys stay in dsh's credential store or your environment. Optional freemodel-auto best-model routing and the guarded default-model switch are both off by default. Version 0.9.0, not yet published to npm (build it from this repo).

How it works

discover ──► filter ──► probe (tiny, cached, budgeted) ──► rank ──► select ──► write configs ──► notify

1. Discover free models from the public OpenRouter catalog (GET /api/v1/models, no key needed). A model counts as free only if its id ends in :free, prompt and completion prices are 0, and it takes and returns text. The openrouter/free meta-router is excluded.
2. Filter by tool-calling support, minimum context window, id globs, and expiry: models whose expiration_date is within N days are dropped (expired models are always dropped, even with exclude_expiring_within_days: null), and models expiring within 30 days are penalised.
3. Probe each candidate with one tiny streaming request (max_tokens: 16 by default, providers.openrouter.probe_max_tokens) using your own OpenRouter key, measuring availability and time-to-first-token. Probing is frugal: candidates are tried in order of promise, results are cached (6 h by default), at most max_tests new probes are made per run (at least 1), and the round stops while keeping reserve_requests of the free requests left today, as reported by free_model_daily_requests.remaining in GET /api/v1/key (UTC day; per OpenRouter's documentation at the time of writing, 50/day below 10 purchased credits and 1000/day above). If the key endpoint reports nothing, a conservative 50/day is assumed against a local UTC-day counter.
   • Account problems are not model failures. A 401, a 402, an OpenRouter platform 429 (free-model per-minute/per-day cap) or a 403 about the account (key permissions, guardrails, budget limits, or a bare 403 with no details) stops the round after that single request; the model is not cached as failed, and Retry-After / X-RateLimit-Reset pause later runs until the reset. A 403 whose error metadata points at the model itself (routing or moderation details, e.g. a free endpoint that refuses this kind of client, or flagged input) is a failure of that model only: it is cached as failed and probing goes on with the next model. Upstream-provider 429s, 408, 5xx and timeouts count as transient failures and are retried after transient_ttl_minutes (30 by default). So do errors that arrive inside the stream after HTTP 200, such as a mid-stream rate limit ("temporarily rate-limited upstream"): probing goes on with the next model; only an explicit 401/402 or account-level 403 in the stream still stops the round. Other errors (e.g. 400/404 "no endpoints") are model failures. On Groq, every 401, 403 and 429 is treated as an account problem.
   • Without an API key, test, rank and sync exit with an error and send nothing.
4. Rank with an explainable score (availability ≫ latency, context, tools, expiry). Fallbacks are models that passed their probe; untested models are used only if you set ranking.fallbacks_allow_untested: true, and are then marked (untested). Non-chat models (safety classifiers, embedding, rerank, speech) are excluded by default.
5. Write the primary model and fallback list into each configured target. Every write is idempotent (unchanged → untouched), supports --dry-run with a diff, makes a timestamped *.bak-* backup (mode 0600 or stricter, the newest 5 kept by default; see backups) and replaces the file atomically. With backups.keep: 0 no backup is made and existing backups are left alone (never deleted); delete old *.bak-* files yourself if you turn backups off. If no model passes its probe, nothing is written.
6. Notify (optional) via Telegram or a generic webhook when the primary model changes. If any target fails to write, the run exits 1, the recorded state is not advanced and no notification is sent; the next run retries.

Install

pipx install freemodel-sync              # latest release from PyPI
pipx install git+https://github.com/ndwlbj/freemodel-sync   # unreleased changes from main
# or run without installing
uvx --from git+https://github.com/ndwlbj/freemodel-sync freemodel-sync --help
# from a local checkout
pipx install .

Requires Python 3.10+ (0.3.3 was the last release that supports 3.9). Runtime dependencies: httpx, PyYAML.

Quick start

export OPENROUTER_API_KEY=...            # your own key; only read from the environment
freemodel-sync discover                  # list free models (no key needed)
freemodel-sync test --max-tests 3        # probe the 3 most promising ones
freemodel-sync rank --json               # full ranking + selection as JSON
cp examples/config.example.yml ~/.config/freemodel-sync/config.yml   # then uncomment and edit targets
freemodel-sync sync --dry-run            # show the diffs
freemodel-sync sync                      # write configs, notify on change

Run sync from cron or a systemd timer, e.g. every 3 hours. Keep the key out of the crontab: put export OPENROUTER_API_KEY=... in a file only you can read (chmod 600 ~/.config/freemodel-sync/env) and source it, or use EnvironmentFile= in a systemd unit.

0 */3 * * * . "$HOME/.config/freemodel-sync/env" && freemodel-sync sync >> "$HOME/.cache/freemodel-sync/sync.log" 2>&1

Exit codes: 0 success, 1 error (bad config, missing key, a target failed), 2 no usable model (no file written), 3 probing was stopped by an account/quota problem such as a platform rate limit or missing credits (no file written, no model marked as failed), 4 another run is still in progress (test, rank and sync take a lock next to the state file, so overlapping cron runs never probe twice). The lock is an advisory OS lock on <state_path>.lock (flock on Linux and macOS): the operating system releases it when the process exits or crashes, so there is no stale lock to delete. It only serializes runs that use the same state_path, does not stop other programs from editing the target files, and may not work on network file systems such as NFS. On Windows the lock uses msvcrt; releases up to 0.3.3 take no lock there. test and rank both return 3 when an account/quota problem stopped probing.

sync --dry-run is zero-cost: it uses only cached probe results (from an earlier test/sync), sends no chat or /key request, needs no API key, and writes no target, state or cache file. It still fetches the public model list. Add --probe to probe as usual during a dry run (this spends free-tier requests and updates the cache). rank --no-probe and sync --no-probe likewise use cached results only.

Configuration

Default location: ~/.config/freemodel-sync/config.yml (override with -c FILE or $FREEMODEL_SYNC_CONFIG). Every key is optional; see examples/config.example.yml for the full annotated file. Its targets and notification channels are commented out: uncomment the ones you use and point them at your own files.

• An unknown key (usually a typo such as testing.max_test) is ignored with a warning that names it. Keys under providers.<name> are provider options and are not checked.
• On/off options (notify.*.enabled, the dsh option check_model, preserve_comments) accept true/false, yes/no, on/off and 1/0, quoted or not. Any other value is an error rather than a guess.
• A target file that does not exist is created with only the managed settings, and a warning is logged (often the path is a typo or a placeholder).
• With no targets, sync writes no file but still records the selection in the state file and sends notifications when it changes (a notify-only setup).

Providers

Groq and multi-provider ranking need freemodel-sync 0.3.0 or newer. SenseNova is new in P7 and Bailian in P8; both are off by default. Without plan: token-plan SenseNova lists and probes nothing, and a model whose catalog price is no longer the string "0" is removed with a warning. Bailian lists and probes nothing until BOTH free_quota_guard: confirmed and a non-empty models allowlist are set; its per-model allowances are entered by hand, labelled "手动，未核实", and never used to deduct the local probe budget. Neither SenseNova nor Bailian is part of the public daily snapshot (both need the user's own account). All enabled providers feed one ranking. The providers section replaces the default (OpenRouter only), so you can drop OpenRouter or set enabled: false. Each provider uses its own key variable, cache file, daily counter and rate-limit pause, and a quota problem stops probing only for that provider. The first enabled provider uses testing.cache_path itself; every other one uses <name>.<provider>.json next to it (e.g. results.groq.json). The binding follows the order of the providers section, so reordering it hands the first provider's daily counter and rate-limit pause to another provider until the next UTC day (cached results are keyed by provider and never mix). providers.<name>.daily_free_limit overrides testing.daily_free_limit for one provider. Fallbacks always come from the primary's provider. Restrict a target to the providers its tool has a key for with options.providers: [openrouter]; otherwise its base URL may switch provider. Details and sources are in docs/providers.md.

Each discovered model carries capabilities (tools, vision, reasoning; null means the catalog does not say, never false) and an optional model-level quota. The shared quota type supports a five-hour rolling window (rolling_5h), a weekly window (week), a daily reset (day), a one-off total (total) and a public-beta marker (beta); missing values stay unknown. rank --json / output.ranking_json writes schema_version: 2 and includes these objects under every model and a normalised quota per provider, while keeping the old daily_free_limit / daily_free_used / daily_free_remaining keys so a v1 reader still works. Old caches and snapshots without the new fields are read as unknown. The dsh plugin shows compact capability/quota badges in its picker, including 能力未知 / 额度未知 for unknown values; a hand-entered allowance is labelled 手动，未核实. A probe can also fail with capacity (queue / no capacity): it is retried like a transient failure, gets its own consecutive counter, is penalised in the ranking, and after ranking.capacity_downgrade_after (default 3) such results in a row it is no longer selectable. A quota failure (Bailian's 403 AllocationQuota.FreeTierOnly) stops that channel until the next UTC day.

providers:
  openrouter: {}
  groq: {plan: free}          # only if your Groq org is on the Free plan
  sensenova: {plan: token-plan}  # only inside the SenseNova Token Plan beta
  bailian:
    free_quota_guard: confirmed  # console "stop when the free quota is used up" switch is on
    models:
      - id: deepseek-v3.2       # example id; use the console's ids
        quota: {unit: tokens, period: total, limit: 1000000}
targets:
  - adapter: openai-env
    path: ~/bot/.env
    options: {providers: [openrouter]}   # this bot only has an OpenRouter key

Manual tier-2 registration (P9)

Platforms without a public API (ZCode, Qoder, WorkBuddy) can be registered by hand in manual_providers. A registration contains only a model id, an optional label/quota note and an expiry date; the Python side never requests those domains. freemodel-sync registry list|check [--json] shows the entries, expiry and expired state using local data only. Manual entries are tier 2, shown as 手动，未核实, and are excluded from automatic selection by default; a target must list the registration in options.providers to select it. Availability probing is not done by Python at all: the dsh plugin does it through ctx.llm, only when manualProbe.enabled and the entry's probe are both on, at most once per model per day. An account error stops that platform for the day; capacity is recorded but does not stop it. See docs/manual-registry.md.

Targets (adapters)

Example dsh result:

- id: agent-default-model
  config: {provider: openrouter, model: 'vendor/some-model:free'}

Known limitations

• AstrBot: both layouts were checked against AstrBot source (v4.29.1); an entry matching neither raises an error instead of being skipped. AstrBot keeps its config in memory and rewrites the file when you save in the WebUI, so restart AstrBot after a sync (and avoid editing in the WebUI at the same time). Provider entries have no fallback list, so only the primary model is written; mapping fallbacks to AstrBot's fallback_provider_ids is not implemented yet. When the model changes, the whole JSON file is written back with 2-space indentation and non-ASCII text kept as is (the format AstrBot itself writes), so a hand-formatted file is reformatted; keys and values other than the model are unchanged.
• YAML comments (0.3.0+): install the optional extra (pipx install "freemodel-sync[yaml-comments]") and openai-yaml edits the file with ruamel.yaml, keeping comments, key order and quoting. Without it, PyYAML re-dumps the whole document when a value changes (an unchanged file is never rewritten): comments are dropped and unrelated values can be re-typed or re-quoted (1.10 becomes 1.1, yes becomes true, 0755 becomes 493, "yes" becomes 'yes'). So with the default preserve_comments: auto and no ruamel.yaml, a file that would lose anything is refused (the target fails and the file is left unchanged); a file PyYAML writes back as is (for example one freemodel-sync already wrote this way) is still updated. preserve_comments: true requires ruamel.yaml; false always uses PyYAML and accepts the re-dump. The dsh adapter always preserves comments.
• Time-based pricing: a :free model whose pricing.overrides has a non-zero price (for example at peak hours or for long contexts) is skipped with a warning.
• Free quota: the daily budget comes from free_model_daily_requests (never inferred from is_free_tier); testing.daily_free_limit overrides it, counted per UTC day. The counters are read once, before probing, and OpenRouter's used count can trail requests that were already sent, so shown quota figures may lag. When this tool's own probe count for the UTC day is higher than the reported used, the budget uses the local count.
• Timeouts: testing.timeout_s applies to probe requests only. Fetching the model list and the quota uses a fixed 30-second timeout.
• Providers: OpenRouter, Groq, SenseNova and Bailian (the last three need an explicit confirmation). Bailian's remaining free allowance has no verified public API yet, so its quota stays manual. SiliconFlow was evaluated but its API does not say which models are free; see docs/providers.md.

Notifications

• Telegram: set notify.telegram and export TELEGRAM_BOT_TOKEN / TELEGRAM_CHAT_ID.
• Webhook (e.g. a QQ bot bridge): set notify.webhook.url_env (default FREEMODEL_SYNC_WEBHOOK_URL). Webhook URLs usually contain a token, so keep them in the environment: an inline notify.webhook.url works but logs a warning, plus another if the config file is readable by other users. The POST body is {"event": "primary_changed", "old", "new", "fallbacks", "text"}.

Notification failures are logged (without tokens or full URLs) and never fail a sync. A configured channel whose environment variables are not set logs a warning when a notification is due, so a misspelt variable name does not go unnoticed; remove the channel (or set enabled: false) if you do not use it.

Daily health check

The repository runs an automated daily health check of free models with GitHub Actions (daily-snapshot.yml).

• When: scheduled with cron 17 1 * * *, i.e. 01:17 UTC, about 09:17 Beijing time (UTC+8). GitHub can start scheduled runs late, or skip them when it is busy; the workflow can also be started by hand.
• What: it discovers the free models of OpenRouter (and of Groq, only when a Groq key is configured and the Groq organization is confirmed to be on the Free plan). It then probes at most 15 models per provider, best static score first, with one tiny request each, staying within the free-request quota and keeping 10 requests in reserve. An account or quota error stops probing for that provider. Models past their expiration date are not probed; the snapshot only lists their ids under the provider.
• Where: results are committed to the public data branch (main is never touched): latest snapshot, plus latest.json and one snapshots/YYYY-MM-DD.json per day in the snapshot folder.
• When a provider fails: if model discovery fails for one provider, that day's file is marked "partial": true and latest.* keep the last complete snapshot, so the public report never shows that provider's models as gone. If discovery fails for every provider, nothing is committed and the run fails.
• Check the date: the snapshot runs with the maintainer's own OpenRouter key. If that key stops working, or GitHub disables the schedule (it does so after 60 days without activity in a public repository), no new snapshot is committed and latest.* simply stays old. The date is in the title of latest.md and in the newest YYYY-MM-DD.json file name: if it is more than two days old, treat the snapshot as stale.

The table shows each model's status (ok, 403, 404, 429, timeout, ...), latency and context length, plus the changes since the previous snapshot and models expiring soon. It is informational only: each result is one request from GitHub Actions at one moment and can differ for you, so run freemodel-sync test with your own key before relying on a model.

Writing a provider

Providers are plugins. Subclass Provider, implement discover() and test() (and optionally quota()), and register it:

from freemodel_sync.providers import Provider, register_provider


@register_provider
class MyProvider(Provider):
    name = "myprovider"
    base_url = "https://api.example.com/v1"
    api_key_env = "MYPROVIDER_API_KEY"

    def discover(self): ...
    def test(self, model, timeout=30.0): ...

Possible future providers (not implemented, no timeline): Zhipu GLM Flash, Google Gemini. Adapters follow the same pattern (freemodel_sync.adapters.base.Adapter + register_adapter).

Development

python -m pip install -e ".[dev]"
ruff check src tests scripts
ruff format --check src tests scripts
pytest

The dsh plugin (lists free OpenRouter/Groq/SenseNova/Bailian models in the dsh picker, with optional freemodel-auto best-model routing and guarded autoDefaultModel profile switch, both default off) lives in plugins/dsh-freemodel/ (TypeScript, pnpm; see its README and docs/dsh-plugin-verification.md).

Tests never touch the network (HTTP is mocked with httpx.MockTransport). See CONTRIBUTING.md.

Security

To report a vulnerability privately, see SECURITY.md.

• Keys are read only from environment variables; freemodel-sync does not write them to config files or print them. Log output (including -v) redacts the exact key values the run read from the environment as well as anything shaped like a key (Telegram bot tokens, Bearer tokens, sk-, sk-or-, sk-ws-/sk-sp- and gsk_ keys), and HTTP request logging (--debug-http) is off by default.
• Symlinked targets are followed: the real file is backed up and replaced, the link is kept.
• Never commit your config.yml, .env files, backups or cache; the .gitignore covers the defaults.

License

MIT © freemodel-sync contributors