ollama-forge

Get models from Hugging Face, convert them, add adapters, and run them in Ollama — without needing deep expertise. One place for fetch, convert, adapters, and recipes.

Install: pip install ollama-forge or uv tool install ollama-forge — PyPI. From this repo: uv sync then uv run ollama-forge; or uv tool install . to use the CLI from anywhere.

Quick start:

ollama-forge fetch TheBloke/Llama-2-7B-GGUF --name my-model && ollama run my-model

Or the shortest path: ollama-forge start --name my-model then ollama run my-model.

Documentation (Wiki)

Detailed guides live in the wiki/:

Why ollama-forge

• One place — Fetch, convert, adapters, recipes; no scattered scripts.
• Simple — Clear commands and docs; try things without being an ML expert.
• Local-first — Get models running in Ollama on your machine.

Setup (one-time)

• Python 3.10+. From PyPI: pip install ollama-forge or uv tool install ollama-forge (PyPI). From repo: uv sync then uv run ollama-forge; use uv tool install . from the repo root to put ollama-forge on your PATH.
• Ollama — Install and ensure ollama is on your PATH.
• Verify: ollama-forge check — see what’s installed. ollama-forge doctor for diagnosis; doctor --fix to apply safe fixes. See Installation for optional llama.cpp (finetune/quantize).
• Optional extras: pip install ollama-forge[net] adds requests for HTTP paths (proxy, security-eval, download-lists); ollama-forge[abliterate] for abliterate run/proxy (see Abliterate).
• Optional: Run Ruff and tests before commit/push: git config core.hooksPath .githooks. See .githooks/README.md. To fix lint before pushing without hooks: ./scripts/lint-fix.sh.

Commands at a glance

Full list: Command Reference. Run ollama-forge --help for options.

Simplest workflows

Beginner (one command):

uv run ollama-forge start --name my-model
ollama run my-model

Uses default model + balanced profile. Use --profile fast|balanced|quality|low-vram and --task chat|coding|creative. See Quick Start.

Auto (any source): Recipe, GGUF path, HF repo, base model, or adapter — the tool detects and runs the right flow:

uv run ollama-forge auto ./recipe.yaml
uv run ollama-forge auto TheBloke/Llama-2-7B-GGUF --name my-model
uv run ollama-forge auto llama3.2 --name my-assistant --system "You are helpful."

See Auto & Plan.

Fetch from Hugging Face: When the repo has GGUF files:

uv run ollama-forge fetch TheBloke/Llama-2-7B-GGUF --name my-model
ollama run my-model

Use --quant Q4_K_M to pick size. For gated or private repos, set HF_TOKEN or run huggingface-cli login. See Fetch & Convert.

Local GGUF: uv run ollama-forge convert --gguf /path/to/model.gguf --name my-model. Optional --quantize Q4_K_M (needs llama.cpp on PATH). See Quantization.

Recipe (one file): uv run ollama-forge build recipe.yaml. See Recipes for format and examples. Sampling options (temperature, top_p, repeat_penalty) are available on fetch, convert, build, and create-from-base (Modelfile, Recipes).

Adapters: adapters search "llama lora", then fetch-adapter <repo> --base <base> --name <name>, or retrain --base <base> --adapter <path> --name <name>. See Adapters.

Training data → model: Validate JSONL, prepare for trainer, generate script: train --data ./data/ --base llama3.2 --name my-model --write-script train.sh. See Training Data and Retrain Pipeline.

Other topics

• Hugging Face repo without GGUF — Convert with llama.cpp first, then convert. Wiki.
• Refusal removal (abliterate) — abliterate compute-dir; optional deps: uv sync --extra abliterate. For agents with tool support use the lightweight proxy: abliterate proxy --name <name>. Wiki.
• Downsizing (distillation) — downsize --teacher <hf> --student <hf> --name <name>. Wiki.
• LLM security evaluation — Run prompt sets against Ollama/serve, score refusal/compliance, get ASR and KPIs: security-eval run <prompt_set>. Optional UI: uv sync --extra security-eval-ui then security-eval ui. Wiki: Security Eval.
• CI — Example GitHub Actions in CI / Automation.