Skip to main content

⚙️ PromptKit

Lint and test your LLM prompts before they reach production.

A typo in a prompt is a bug. PromptKit catches it like one.

PyPI Python CI mypy strict License

Documentation · Getting started · Issues

Prompts as f-stringsPrompts as data
prompt = f"""You are a support agent
for {product}. Be concise.

Customer wrote: {message}
"""
if order_id:
    prompt += f"Order: {order_id}"

msg = {"role": "user", "content": prompt}
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[msg],
)
text = resp.choices[0].message.content

Logic tangled with copy. A typo in {prodcut} ships silently. No system role, no review, no tests, no idea what it costs until the invoice arrives.

name: support_reply
description: Support reply, house style
version: 1.0.0
messages:
  - role: system
    template: |
      You are a support agent for
      {{ product }}.
      {% include '_partials/style.j2' %}
  - role: user
    template: |
      Customer wrote: {{ message }}
input_schema:
  product: str
  message: str

Reviewable in a pull request. A typo is a lint error. Real roles, a shared house style, and promptkit cost before you spend anything.

Install

pip install 'promptkit-core[openai]'      # or [anthropic] · [ollama] · [all]

The base install ships no HTTP client and no provider SDK. Loading, rendering, validation, composition, linting and cost estimation all work with nothing network-shaped in your dependency tree.

Catch the typo before you pay for it

Fat-finger it — {{ prodcut }} in the template while the schema still says product. An f-string would ship that to production. Here it does not get past the gate:

$ promptkit lint support_reply.yaml
support_reply
  PK001 'prodcut' is used but not declared (prodcut)
  PK002 'product' is declared but never used (product)

2 finding(s)

Exit code 1, so CI stops. Fix it, and the rest of the loop is free and offline:

$ promptkit lint support_reply.yaml
✓ support_reply

$ promptkit render support_reply.yaml --set product=Acme --messages
system
You are a support agent for Acme.
Answer in plain language. Prefer short sentences.

user
Customer wrote: placeholder

$ promptkit cost support_reply.yaml --model gpt-4o-mini
support_reply with gpt-4o-mini
Input tokens: 21 (exact)
Output tokens: 500 (assumed)
Estimated cost: $0.000303

Anything you leave out is filled with a placeholder, so you can render and price a prompt before you have real inputs. Not one of those commands needed an API key.

From Python

from promptkit import load_prompt, run_prompt
from promptkit.engines.openai import OpenAIEngine

prompt = load_prompt("support_reply.yaml")

with OpenAIEngine() as engine:
    completion = run_prompt(prompt, {"product": "Acme", "message": msg}, engine)

completion.text  # the reply
completion.usage.prompt_tokens  # 241, straight from the provider
completion.usage.estimated  # False — measured, never quietly guessed
engine.cost_of(completion)  # 8.895e-05

Need JSON back? Hand it a Pydantic model and get a validated object, with automatic re-prompting when the model gets it wrong:

result = run_structured(prompt, inputs, engine, output_model=Invoice)
result.value.total  # a float, and your type checker knows it

What you get

Catch it before you pay Nine lint rules with codes — undeclared variables, unused schema fields, unresolved includes. --strict and --format json for CI
Evals that gate a build contains, regex, json_schema, max_cost, max_latency, and an LLM judge. JUnit output, concurrent, non-zero on failure
Honest costing Token counts come from the provider. Usage.estimated says when a number was guessed, so a cost report can never quietly lie
One house style Share text across prompts with {% include %}. Edit the partial and every dependent prompt changes fingerprint, so caches invalidate correctly
A prompt is not code Templates render in a Jinja sandbox; includes resolve through a confined loader. Traversal, symlink escapes and __class__ tricks are refused, with tests to prove it
Identity that means something Prompts are identified by a fingerprint over messages, schemas and resolved includes — not a version string someone forgot to bump
Providers, plugged in OpenAI, Anthropic, Ollama, and any OpenAI-compatible endpoint. One error hierarchy, so except RateLimitError means the same thing everywhere
Ship your own engine Register through the promptkit.engines entry point from your own package. No PR to this repo

The CLI

run Render and send. --stream, --structured
render Render locally. Free, offline, no key
lint Nine rules. --strict, --format json
test Run eval suites. --format junit
diff Compare two prompts by fingerprint
info · list Inspect one prompt, or a whole tree
cost Token counts and cost before you spend
init · engines Scaffold a prompt · see what's installed

Variables come from --set key=value, --vars '{"json": true}', or --vars-file. --set coerces to the type your schema declares.

Not an agent framework

Tool calling, agent loops, conversation memory and RAG are permanently out of scope — a boundary, not a gap. Nothing traps you: engine.client is the real SDK client and completion.raw the real provider response, so dropping down is one attribute away.

Coming from 0.1.x

Your prompt files load unchanged — asserted per-file by the test suite. Most code needs two edits, and a codemod handles the mechanical ones:

python -m promptkit.codemod your_package/          # dry run, prints a diff
python -m promptkit.codemod your_package/ --write  # apply it

Details in the upgrade guide.


promptkit-core.ochotzas.com

Prompt files · Schemas · Composition · Structured output · Evaluation · Writing an engine

MIT licensed · Contributing · Architecture

If PromptKit saves you a debugging session, a ⭐ helps others find it.

Metadata

Release files for promptkit-core 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for promptkit-core 1.0.0
File Size Uploaded Provenance Actions
promptkit_core-1.0.0.tar.gz 99.6 kB With provenance Details for promptkit_core-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for promptkit-core 1.0.0
File Interpreter ABI Platform Actions
promptkit_core-1.0.0-py3-none-any.whl Python 3 none any Details for promptkit_core-1.0.0-py3-none-any.whl

Total release size: 180.4 kB

Release files / promptkit_core-1.0.0.tar.gz

Download URL promptkit_core-1.0.0.tar.gz
Size 99.6 kB
Tags Source
SHA-256 checksum
How to use checksums
0b23b9d09d2892ff49c234bbf32ce3a1215d5209101af2c08c3f385988f47ab4
BLAKE2b-256 checksum
How to use checksums
c5db9833dcf1aeafc2b7e5d35215b3adf66833b777dfe0bc507125164ba03ddd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 16, 2026.

Transparency log

Release files / promptkit_core-1.0.0-py3-none-any.whl

Download URL promptkit_core-1.0.0-py3-none-any.whl
Size 80.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
93c7a4e5dd578612e6deb6dbc607c4f6d940ddb8a163a05fcb3243faf2083bc5
BLAKE2b-256 checksum
How to use checksums
e4b2a9055258693c779fa48e262b013060c6994946d5c6095523a3ef94f436f0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 16, 2026.

Transparency log

Security for promptkit-core 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 2 files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for promptkit_core-1.0.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 16, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page