basemode

Make any LLM do raw text continuation.

basemode coerces chat-tuned models into clean next-token continuation mode (instead of assistant-style replies), with strategy selection handled per model/provider.

Install

pip install basemode

Set provider keys via environment variables or .env (for example OPENAI_API_KEY, ANTHROPIC_API_KEY, OPENROUTER_API_KEY, GEMINI_API_KEY, GROQ_API_KEY, TOGETHER_API_KEY).

Quickstart

# Single continuation (default model if configured, else fallback)
basemode "The ship rounded the headland and"

# Parallel continuations
basemode "The ship rounded the headland and" -n 3

# Inspect selected strategy and pricing metadata
basemode info claude-sonnet-4-6

# Rank coercion strategies for a model by how cleanly they continue text
basemode bench claude-sonnet-4-6

# Show only key-configured models
basemode models --available

CLI

basemode --help
basemode run --help
basemode models --help
basemode info --help
basemode strategies --help

Useful commands:

• basemode run (default): stream continuation text
• basemode models: list models (supports --verified and --json for picker UIs)
• basemode providers: list provider IDs
• basemode info: show normalized model + prompt strategy (and where that choice came from) + pricing metadata
• basemode bench: rank coercion strategies for a model, and pin the winner
• basemode default: get/set your default model
• basemode keys: manage stored API keys

Choosing a strategy

Coercion either works or it doesn't, and basemode scores which. basemode bench runs each strategy against the model over four probe prefixes — prose, technical, poetry, dialogue — and scores every result for assistant behavior: preamble, refusal, echoed prefix, chat turns, stray formatting.

basemode bench claude-opus-5 --samples   # compare
basemode bench claude-opus-5 --save      # pin the winner for this model

┏━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ Strategy ┃ Score ┃ Flags / error        ┃ Mean s ┃
┡━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ few_shot │ 0.93  │ quoted               │ 2.07   │
│ system   │ 0.75  │ empty                │ 2.24   │
│ prefill  │ 0.00  │ BadRequestError: ... │ 0.30   │
└──────────┴───────┴──────────────────────┴────────┘

At runtime the strategy comes from the first of: an explicit --strategy, a local pin, the prompt method verified for that model in the table below, then model-name heuristics. basemode info MODEL shows which one applied.

Server

basemode serve exposes an OpenAI-completions-compatible POST /v1/completions endpoint backed by continue_text/branch_text, so llama.cpp-style clients (e.g. Tapestry Loom) can drive basemode's model-coerced continuations as a local backend instead of needing a locally-hosted base model.

pip install 'basemode[server]'
basemode serve --port 8080

Point a client's "OpenAI-style Completions" endpoint at http://127.0.0.1:8080/v1/completions. Requests accept model, prompt, max_tokens, temperature, n (parallel branches), echo, and strategy; stream is not supported (responses are always synchronous JSON). No logprobs are returned, since basemode routes through chat-completion coercion rather than a real base model.

Python API

from basemode import continue_text, branch_text

async for token in continue_text(
    "The ship rounded the headland and",
    model="gpt-4o-mini",
    max_tokens=120,
):
    print(token, end="", flush=True)

async for idx, token in branch_text(
    "The ship rounded the headland and",
    model="gpt-4o-mini",
    n=3,
    max_tokens=80,
):
    print(idx, token, end="", flush=True)

Docs

Full docs are in docs/ and can be served with MkDocs:

make docs-serve

Then open http://localhost:8001.

Integration Health Checks

Run live provider checks (real APIs, key-aware skips):

uv run pytest -m integration tests/test_integration.py -q

This writes a machine-readable report to dist/integration/provider_health.json with per-model status, latency, token estimates, and estimated USD cost.

Verified Models

Single generated table, refreshed by CI.

Legend: ✓ = LiteLLM pricing present and release date available; ⚠ = missing/approximate field or known issue.