owist-modelfile-lint

Static validation for Ollama Modelfiles. Catch broken FROM paths, invalid PARAMETER values, and missing TEMPLATEs before you run ollama create and get a cryptic Go error three minutes into a model build.

Built by Openwist AI, maker of the LimitAI open model family.

The problem

ollama create parses your Modelfile on the Go side and fails late:

Error: invalid file magic

That's it. No line number, no hint about which instruction caused it, and you find out only after Ollama has already started reading your (possibly multi-gigabyte) model file. A typo'd PARAMETER key gets silently ignored instead of erroring. A missing TEMPLATE on an unrecognized base model ships a model with no chat formatting at all, and you don't notice until it responds with garbage.

owist-modelfile-lint reads your Modelfile before any of that, the same way ruff or eslint check source code before you run it.

Install

pip install owist-modelfile-lint

Usage

CLI

modelfile-lint ./Modelfile

[ERROR]   line 1: FROM path './sophia-q4.gguf' does not exist  (FROM005)
[ERROR]   line 5: PARAMETER 'temprature' is not a recognized Ollama parameter (did you mean 'temperature'?)  (PARAM002)
[WARNING] line 7: PARAMETER 'temperature' value 5.7 is outside the typical range [0.0, 2.0] (default: 0.8) — this is valid syntax but likely unintentional  (PARAM007)
[WARNING] general: no TEMPLATE instruction found and FROM points to a local file/directory — Ollama's chat-template auto-detection may fail for unrecognized architectures, producing a model with no chat formatting at all. Consider adding an explicit TEMPLATE.  (TPL002)
[INFO]    line 1: estimated memory at context=8192: 4.10 GB weights + 1.00 GB KV cache + 0.75 GB overhead = ~5.85 GB total  (RES003)
[WARNING] line 1: estimated 5.85 GB exceeds detected free VRAM (4.0 GB on RTX 3060) — Ollama will offload some layers to CPU, reducing speed  (RES005)
[INFO]    line 1: estimated decode speed on RTX 3060: 43.2-75.6 tok/s (physics-based range, not a measured benchmark)  (RES008)

✗ ./Modelfile: 2 error(s), 4 warning(s)

Exit code is 0 when there are no errors (warnings don't fail the check), 1 otherwise — so it's a drop-in CI or pre-commit gate:

modelfile-lint ./Modelfile || exit 1

Other flags:

modelfile-lint ./Modelfile --quiet      # errors only, suppress warnings
modelfile-lint ./Modelfile --json       # machine-readable output
modelfile-lint ./Modelfile --no-color
modelfile-lint ./Modelfile --no-estimate  # skip memory/speed estimation (syntax-only, faster for CI)

Memory and speed estimation (new in v0.2.0)

When FROM points at a local .gguf file, owist-modelfile-lint now estimates whether the model will actually fit on your hardware and roughly how fast it'll run — before you wait through a multi-gigabyte ollama create to find out.

[INFO]    line 1: estimated memory at context=8192: 4.10 GB weights + 1.00 GB KV cache + 0.75 GB overhead = ~5.85 GB total  (RES003)
[WARNING] line 1: estimated 5.85 GB exceeds detected free VRAM (4.0 GB on RTX 3060) — Ollama will offload some layers to CPU, reducing speed  (RES005)
[INFO]    line 1: estimated decode speed on RTX 3060: 43.2-75.6 tok/s (physics-based range, not a measured benchmark)  (RES008)

Two very different confidence levels here, and the output says so explicitly:

• Memory is a real calculation — weights size (on-disk, quantization already baked in) + KV cache (from the model's own architecture metadata and your PARAMETER num_ctx) + a documented compute-buffer allowance. Not a guess.
• Decode speed is a physics-grounded range (memory-bandwidth-bound at batch size 1), not a fabricated precise number — no static tool can know your exact runtime conditions, and this one doesn't pretend to.

Detects NVIDIA GPUs (nvidia-smi) and Apple Silicon (sysctl) automatically; falls back to system RAM otherwise. Known gaps, stated plainly: AMD GPUs aren't detected yet, and pull-by-name models (FROM llama3.2) can't be inspected — only local .gguf file paths, since there's nothing on disk yet to read metadata from. See CHANGELOG.md for the full list.

RAM detection uses os.sysconf on Linux/macOS — no extra dependency. Windows needs one optional extra for this specific check:

pip install "owist-modelfile-lint[resources]"

Python API

from owist_modelfile_lint import lint

result = lint("Modelfile")

if not result.ok:
    for issue in result.issues:
        print(issue)
    raise SystemExit(1)

from owist_modelfile_lint import lint_text

# lint content that doesn't exist on disk yet, e.g. generated programmatically
result = lint_text("""
FROM llama3.2
PARAMETER temperature 0.7
SYSTEM You are a helpful assistant.
""")
print(result.ok)  # True

LintResult gives you .ok, .issues, .errors, .warnings, .infos, and is truthy/falsy based on .ok so if result: works too.

What it checks

This is static analysis — it never loads model weights or runs Ollama. Every check, including the new resource estimation, reads only the GGUF header and metadata section — a few KB at most — never the multi-gigabyte tensor data itself, so it stays fast even on 70B-class files.

What it deliberately does not do

• It does not validate that your TEMPLATE Go-template syntax is semantically correct for the model's actual chat format — that requires knowing what the base model expects, which is out of scope for a static linter.
• It does not check model quality — see tinyeval (planned) for that.
• It does not talk to the Ollama daemon or registry. Library model references like FROM llama3.2 are accepted as-is without checking whether that tag exists.
• It does not benchmark or guarantee speed numbers — decode-speed estimates are a physics-based range (memory bandwidth ÷ model size), not a measurement. Real speed depends on kernel implementation, thermal state, and background load, none of which a static linter can see.

Why "owist"

Short for Openwist AI — we build the LimitAI open model family (Anan, Sophia) and got tired of debugging our own Modelfiles by trial and error.

License

MIT