⚙️ PromptKit
Lint and test your LLM prompts before they reach production.
A typo in a prompt is a bug. PromptKit catches it like one.
| Prompts as f-strings | Prompts as data |
|---|---|
prompt = f"""You are a support agent
for {product}. Be concise.
Customer wrote: {message}
"""
if order_id:
prompt += f"Order: {order_id}"
msg = {"role": "user", "content": prompt}
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[msg],
)
text = resp.choices[0].message.content
Logic tangled with copy. A typo in
|
name: support_reply
description: Support reply, house style
version: 1.0.0
messages:
- role: system
template: |
You are a support agent for
{{ product }}.
{% include '_partials/style.j2' %}
- role: user
template: |
Customer wrote: {{ message }}
input_schema:
product: str
message: str
Reviewable in a pull request. A typo is
a lint error. Real roles, a shared
house style, and |
Install
pip install 'promptkit-core[openai]' # or [anthropic] · [ollama] · [all]
The base install ships no HTTP client and no provider SDK. Loading, rendering, validation, composition, linting and cost estimation all work with nothing network-shaped in your dependency tree.
Catch the typo before you pay for it
Fat-finger it — {{ prodcut }} in the template while the schema still says product.
An f-string would ship that to production. Here it does not get past the gate:
$ promptkit lint support_reply.yaml
support_reply
PK001 'prodcut' is used but not declared (prodcut)
PK002 'product' is declared but never used (product)
2 finding(s)
Exit code 1, so CI stops. Fix it, and the rest of the loop is free and offline:
$ promptkit lint support_reply.yaml
✓ support_reply
$ promptkit render support_reply.yaml --set product=Acme --messages
system
You are a support agent for Acme.
Answer in plain language. Prefer short sentences.
user
Customer wrote: placeholder
$ promptkit cost support_reply.yaml --model gpt-4o-mini
support_reply with gpt-4o-mini
Input tokens: 21 (exact)
Output tokens: 500 (assumed)
Estimated cost: $0.000303
Anything you leave out is filled with a placeholder, so you can render and price a prompt before you have real inputs. Not one of those commands needed an API key.
From Python
from promptkit import load_prompt, run_prompt
from promptkit.engines.openai import OpenAIEngine
prompt = load_prompt("support_reply.yaml")
with OpenAIEngine() as engine:
completion = run_prompt(prompt, {"product": "Acme", "message": msg}, engine)
completion.text # the reply
completion.usage.prompt_tokens # 241, straight from the provider
completion.usage.estimated # False — measured, never quietly guessed
engine.cost_of(completion) # 8.895e-05
Need JSON back? Hand it a Pydantic model and get a validated object, with automatic re-prompting when the model gets it wrong:
result = run_structured(prompt, inputs, engine, output_model=Invoice)
result.value.total # a float, and your type checker knows it
What you get
| Catch it before you pay | Nine lint rules with codes — undeclared variables, unused schema fields, unresolved includes. --strict and --format json for CI |
| Evals that gate a build | contains, regex, json_schema, max_cost, max_latency, and an LLM judge. JUnit output, concurrent, non-zero on failure |
| Honest costing | Token counts come from the provider. Usage.estimated says when a number was guessed, so a cost report can never quietly lie |
| One house style | Share text across prompts with {% include %}. Edit the partial and every dependent prompt changes fingerprint, so caches invalidate correctly |
| A prompt is not code | Templates render in a Jinja sandbox; includes resolve through a confined loader. Traversal, symlink escapes and __class__ tricks are refused, with tests to prove it |
| Identity that means something | Prompts are identified by a fingerprint over messages, schemas and resolved includes — not a version string someone forgot to bump |
| Providers, plugged in | OpenAI, Anthropic, Ollama, and any OpenAI-compatible endpoint. One error hierarchy, so except RateLimitError means the same thing everywhere |
| Ship your own engine | Register through the promptkit.engines entry point from your own package. No PR to this repo |
The CLI
run |
Render and send. --stream, --structured |
render |
Render locally. Free, offline, no key |
lint |
Nine rules. --strict, --format json |
test |
Run eval suites. --format junit |
diff |
Compare two prompts by fingerprint |
info · list |
Inspect one prompt, or a whole tree |
cost |
Token counts and cost before you spend |
init · engines |
Scaffold a prompt · see what's installed |
Variables come from --set key=value, --vars '{"json": true}', or --vars-file.
--set coerces to the type your schema declares.
Not an agent framework
Tool calling, agent loops, conversation memory and RAG are permanently out of scope —
a boundary, not a gap. Nothing traps you: engine.client is the real SDK client and
completion.raw the real provider response, so dropping down is one attribute away.
Coming from 0.1.x
Your prompt files load unchanged — asserted per-file by the test suite. Most code needs two edits, and a codemod handles the mechanical ones:
python -m promptkit.codemod your_package/ # dry run, prints a diff
python -m promptkit.codemod your_package/ --write # apply it
Details in the upgrade guide.
Prompt files · Schemas · Composition · Structured output · Evaluation · Writing an engine
MIT licensed · Contributing · Architecture
If PromptKit saves you a debugging session, a ⭐ helps others find it.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file promptkit_core-1.0.0.tar.gz.
File metadata
- Download URL: promptkit_core-1.0.0.tar.gz
- Upload date:
- Size: 99.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0b23b9d09d2892ff49c234bbf32ce3a1215d5209101af2c08c3f385988f47ab4
|
|
| MD5 |
c05be3ce24e4d2c0f9c3f00bb50486e4
|
|
| BLAKE2b-256 |
c5db9833dcf1aeafc2b7e5d35215b3adf66833b777dfe0bc507125164ba03ddd
|
Provenance
The following attestation bundles were made for promptkit_core-1.0.0.tar.gz:
Publisher:
cd.yml on ochotzas/promptkit-core
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
promptkit_core-1.0.0.tar.gz -
Subject digest:
0b23b9d09d2892ff49c234bbf32ce3a1215d5209101af2c08c3f385988f47ab4 - Sigstore transparency entry: 2863887612
- Sigstore integration time:
-
Permalink:
ochotzas/promptkit-core@36d7f0240a6a4b02e04a04b2b9821c7a6a06fd28 -
Branch / Tag:
refs/tags/v1.0.0 - Owner: https://github.com/ochotzas
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
cd.yml@36d7f0240a6a4b02e04a04b2b9821c7a6a06fd28 -
Trigger Event:
release
-
Statement type:
File details
Details for the file promptkit_core-1.0.0-py3-none-any.whl.
File metadata
- Download URL: promptkit_core-1.0.0-py3-none-any.whl
- Upload date:
- Size: 80.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
93c7a4e5dd578612e6deb6dbc607c4f6d940ddb8a163a05fcb3243faf2083bc5
|
|
| MD5 |
6c5fc3a83fa09adec7e4c683a8525143
|
|
| BLAKE2b-256 |
e4b2a9055258693c779fa48e262b013060c6994946d5c6095523a3ef94f436f0
|
Provenance
The following attestation bundles were made for promptkit_core-1.0.0-py3-none-any.whl:
Publisher:
cd.yml on ochotzas/promptkit-core
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
promptkit_core-1.0.0-py3-none-any.whl -
Subject digest:
93c7a4e5dd578612e6deb6dbc607c4f6d940ddb8a163a05fcb3243faf2083bc5 - Sigstore transparency entry: 2863887678
- Sigstore integration time:
-
Permalink:
ochotzas/promptkit-core@36d7f0240a6a4b02e04a04b2b9821c7a6a06fd28 -
Branch / Tag:
refs/tags/v1.0.0 - Owner: https://github.com/ochotzas
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
cd.yml@36d7f0240a6a4b02e04a04b2b9821c7a6a06fd28 -
Trigger Event:
release
-
Statement type: