AI Template Runner

Small local models are cheap and fast — they also lose structure the moment a prompt gets long. ait turns that weakness into a workflow: each template is a single Markdown file that gathers context, prompts the model, validates the answer and renders the result. Deterministic scaffolding around a dumb-but-fast model, even one running on your phone.

┌ template file ─────────────────────────────────┐
│ ```toml  →  model, endpoint, sampling          │
│ ```                                            │
│ ---    →  system prompt                        │
│ user prompt:  {cmd.version} output + question  │
│ [test] / [out] →  validate + render output     │
└────────────────────────────────────────────────┘

One file, one command, one predictable answer — no agent framework needed.

Why it matters: a small model cannot keep a workflow in its head, but it can fill in a well-shaped slot. Templates put the workflow in the file so the model only has to do the part it is good at. Failures become visible and reproducible through [test], instead of vague "the model is dumb" sessions.

Quick start

Point ait at any OpenAI-compatible chat endpoint (llama.cpp server, Ollama, LM Studio, ...):

1. Start a model server, e.g. llama.cpp:

llama-server -hf unsloth/Qwen3-0.6B-GGUF --port 8080

2. Run a ready template:

uvx ai-template-runner run TEMPLATE "What is Python?"

Or clone the repository and try it against a fixture without a model at all — the examples/simulation/ templates run against fixed responses, so you can feel the [test]/[out] mechanics in seconds:

git clone https://github.com/brunodavi/ai-template-runner.git
cd ai-template-runner
uvx ai-template-runner run examples/simulation/conventional-commits/feat.md --test

Running on a phone? See docs/termux.md.

Installation

Requires Python 3.12+. The CLI is ait (or ai-template-runner); the package has zero runtime dependencies.

Usage

ait run examples/features/full.md "What is Python?"
ait run examples/features/full.md "What is Python?" --quiet
ait run examples/simulation/conventional-commits/feat.md --test
ait --help
ait run --help

• -q / --quiet: display only the AI response (stdout stays clean for pipes).
• -t / --test: validate the response against the template's [test] table. PASS/FAIL lines go to stderr and the exit code is 1 when any check fails.

Templates

Templates combine TOML configuration and Markdown prompts in a single file. The configuration is defined in a ```toml code block at the beginning of the file, and is parsed and removed from the prompt content. Two --- separators delimit an optional system prompt; everything after them becomes the user prompt. Templates can also execute terminal commands and inject their output directly into prompts.

```toml
[provider]
base_url = 'http://localhost:8080/v1/chat/completions'
model = 'lfm230m'
temperature = 0.2
top_p = 0.9
top_k = 40
min_p = 0.05
max_tokens = 220
seed = 42
stop = []
frequency_penalty = 0.0
presence_penalty = 0.0
repeat_penalty = 1.1

[provider.extras]
typical_p = 1.0
repeat_last_n = 64

[cmd]
message = 'echo hello'
```

---
You are a concise assistant. Use the command output as context.
---

Command output:
{cmd.message}

Answer the following question concisely: {question}

The TOML block is parsed and removed from the prompt content. The optional system prompt is delimited by two --- separators; everything after the second separator is the user prompt. When the separators are omitted, the whole content after the configuration block is the user prompt.

The {question} placeholder is replaced with the CLI input. The {cmd.message} placeholder executes the configured command and inserts its output into the prompt. Placeholders can be used in both the system and user prompts.

The provider supports these generation parameters:

• temperature, top_p, top_k, min_p: control sampling.
• max_tokens: limits the generated response length.
• seed: sets the random seed when supported.
• stop: defines stop sequences.
• frequency_penalty, presence_penalty, repeat_penalty: control repetition.
• extras: passes additional parameters to the API, such as llama.cpp-specific options.

base_url and model are required. Parameter support depends on the API and model server being used.

A template can inherit configuration, commands, and prompts from another template with a top-level extends key placed before any TOML table:

extends = 'base.md'

[provider]
temperature = 0.2

The extends path is resolved relative to the template file itself, so examples/features/ templates use extends = '../base.md'. Child values win per key; nested tables such as [provider.extras] and [cmd] are merged key by key. Empty child prompts fall back to the parent prompts, which lets a template reuse only the parent prompts and configuration.

A template can validate its own output with a [test] table and the --test flag:

[test]
type = 'feat'                              # JSON output field equality
message = 'add health check endpoint'      # exact value comparison
"user.name" = 'Alice'                      # nested JSON path (quoted dotted key)

text = 'exact raw output'                  # exact raw text comparison
text_contains = ['GET_health']             # every fragment present in the raw text
text_regex = '^\{'                         # regex search on the raw text

Run with validation:

ait run examples/simulation/conventional-commits/feat.md --test

PASS/FAIL lines go to stderr (stdout stays clean for pipes) and the exit code is 1 when any check fails. Missing JSON fields and invalid JSON output fail instead of raising. The [test] table participates in extends inheritance like the other config tables.

A template can also render its own display output with an [out] table. The text key is a render template formatted with the parsed JSON fields ({out.type}, {out.message}, ...); missing fields render as empty:

[out]
text = '{out.type}: {out.message}'

Transforms run in two channels that never share the field namespace (out.count is always the JSON field, never str.count):

[out]
text = '{out.message:strip} | {out.message:replace:GET:SET} | {out.message[:4]}'

[[out.upper]]                               # one step: str.upper() with no args

[[out.replace]]                             # named args match the real str signature
old = 'GET_health'
new = 'GET'

[[out.slice]]                               # helper: builtin slice(start, end)
start = 1
end = 10

The [[out.<method>]] tables run in real document order (one table = one step, an empty table means no arguments) and apply to the rendered text — or to the raw response when there is no text. Steps are real str methods, with named arguments mapped positionally by the real signature. The inline spec is a single step from method:arg1:arg2 (argument values may not contain ':'), and {out.field[1:]} follows pythonic slice syntax. Without [out] (or an empty table) the raw AI response is printed. The [out] config participates in extends inheritance per key: the child text and step lists replace the parent's.

The same inline transforms work on every interpolated value in the prompts — {question:strip}, {question[:80]:lower} and {cmd.message:upper}, {cmd.status[1:20]} — since the value stays lazy for {cmd.*} (the command only runs when a field is actually used).

Examples and docs

See examples/base.md for the shared base used by the examples, examples/features/ for one example per feature, examples/ for ready-to-use task templates such as email-subject.md and conventional-commits.md, and examples/simulation/ for simulations that run task templates against fixed fixtures.

For deeper material, read docs/architecture.md (why the code is shaped the way it is), docs/termux.md (running local models on a phone) and docs/maintenance.md (release routine for maintainers).

Development

git clone https://github.com/brunodavi/ai-template-runner.git
cd ai-template-runner
uv sync
uv pip install -e .

Editable installation (-e .) lets you run ait directly while developing, so source changes are available without reinstalling.

Run a template to test your changes:

ait run examples/features/full.md "What is Python?"

Run the test suite (from the repository root):

uv run python -m unittest

Contributions are welcome — CI runs the suite on Python 3.12, 3.13 and 3.14, so that is the floor.

Status

AIT is in active development for local AI experimentation and lightweight, template-driven workflows. Expect the template syntax to be stable and everything around it to keep evolving. See CHANGELOG.md for release history.

Licensed under the MIT license.