promptkit

The Python CLI for prompt engineering — version control, testing, linting, and CI/CD for LLM prompts.

No Docker. No infrastructure. No TypeScript. Just pip install promptkit.

$ promptkit save my-prompt -c "You are a helpful coding assistant. Always respond in {{language}}."
✓ Saved my-prompt v1 (a3f8c2d1)
  Variables: language

$ promptkit lint my-prompt
Score: 95/100 — PASSED

$ promptkit get my-prompt --var language=Python
You are a helpful coding assistant. Always respond in Python.

Why promptkit?

Every prompt management tool out there requires Docker, PostgreSQL, ClickHouse, or a TypeScript runtime. They're designed for platform teams, not individual developers.

promptkit is different:

Feature	promptkit	Langfuse	Promptfoo	Helicone
Install	pip install	Docker + PostgreSQL	npm install	Docker + ClickHouse
Language	Python	TypeScript	TypeScript	TypeScript
Infra needed	None	Heavy	Medium	Heavy
Version control	Git-like, built-in	Database	Files	Database
Prompt linting	✅ 12 rules	❌	❌	❌
Offline-first	✅	❌	✅	❌
Cost estimation	✅	✅	❌	✅

Installation

pip install promptkit

With LLM evaluation support:

pip install promptkit[eval]

Quick Start

Initialize a project

promptkit init

This creates a .promptkit/ directory in your project:

.promptkit/
├── config.json    — Configuration
├── prompts/       — Prompt storage
├── tests/         — Test definitions
├── cache/         — Temporary files
└── .gitignore

Save prompts with version control

# Inline content
promptkit save my-prompt -c "You are a helpful assistant."

# From a file
promptkit save my-prompt -f system-prompt.md -m "Updated instructions"

# From stdin (piping)
cat prompt.md | promptkit save my-prompt

Retrieve prompts

# Latest version
promptkit get my-prompt

# Specific version
promptkit get my-prompt -v 2

# By tag
promptkit get my-prompt -t prod

# With variable substitution
promptkit get my-prompt --var name=Alice --var count=5

# Raw output (no formatting, for piping)
promptkit get my-prompt --raw | pbcopy

Version history

promptkit history my-prompt

┌─────────────────────────────────┐
│ History: my-prompt              │
├─────────┬──────────┬────────────┤
│ Version │ Hash     │ Message    │
├─────────┼──────────┼────────────┤
│ v1      │ a3f8c2d1 │ Version 1  │
│ v2      │ 7b2e9f4a │ Added tone │
│ v3      │ c1d5e8f2 │ Production │
└─────────┴──────────┴────────────┘

Tagging

# Tag latest version
promptkit tag my-prompt prod

# Tag specific version
promptkit tag my-prompt staging -v 2

# Remove a tag
promptkit untag my-prompt staging

Compare versions

promptkit diff my-prompt 1 2

Lint prompts

promptkit lint my-prompt

12 built-in lint rules:

Rule	What it checks
no-empty	Prompt is not empty
min-length	Minimum content length
max-length	Maximum content length
no-trailing-whitespace	No trailing whitespace
no-double-spaces	No double spaces
has-instruction	Contains clear instructions
no-typo-variables	No single-brace {var} typos
balanced-braces	Matching {{ }} and {% %}
no-todo	No TODO/FIXME/HACK markers
has-context	Sufficient context provided
no-ambiguity	No vague language
consistent-format	Consistent markdown formatting

Test prompts

Create a test file (tests.json):

{
  "tests": [
    {
      "name": "renders-correctly",
      "variables": {"name": "Alice"},
      "assertions": [
        {"type": "contains", "value": "Alice"},
        {"type": "not_contains", "value": "{{"},
        {"type": "renders_without_error", "value": null}
      ]
    },
    {
      "name": "no-pii-leak",
      "assertions": [
        {"type": "no_pii", "value": null}
      ]
    },
    {
      "name": "token-budget",
      "assertions": [
        {"type": "token_count_below", "value": 1000}
      ]
    }
  ]
}

Run tests:

promptkit test my-prompt tests.json

13 built-in assertion types:

Assertion	Description
contains	Content contains value
not_contains	Content doesn't contain value
starts_with	Content starts with value
ends_with	Content ends with value
matches_regex	Content matches regex pattern
min_length	Minimum character length
max_length	Maximum character length
has_variable	Template has {{ variable }}
renders_without_error	Jinja2 renders cleanly
token_count_below	Under token budget
no_pii	No PII (emails, SSNs, phones, cards)
json_valid	Specifies JSON output format
word_count_between	Word count in range

Cost estimation

promptkit cost my-prompt
promptkit cost my-prompt -m claude-sonnet-4-20250514
promptkit cost my-prompt -o 1000  # estimated output tokens

Import / Export

# Export a prompt to a file
promptkit export my-prompt output.md

# Import a prompt from a file
promptkit import my-prompt input.md -m "From production"

Statistics

promptkit stats

Jinja2 Templates

promptkit uses Jinja2 for variable substitution:

You are a {{ role }} assistant.

Help the user with {{ task }}.

{% if formal %}
Use formal language.
{% endif %}

Respond in {{ language | default("English") }}.

Python API

Use promptkit programmatically:

from promptkit import Config, PromptStore, PromptLinter, PromptTester, TestCase

# Initialize
config = Config()
store = PromptStore(config)

# Save a prompt
version = store.commit("my-prompt", "You are a helpful assistant.")

# Get prompt content
content = store.get("my-prompt")
content = store.get("my-prompt", tag="prod")
content = store.get("my-prompt", variables={"name": "Alice"})

# Lint
linter = PromptLinter()
report = linter.lint_content("You are a helpful assistant.")
print(f"Score: {report.score}/100")

# Test
tester = PromptTester()
cases = [
    TestCase(
        name="basic-check",
        assertions=[
            {"type": "contains", "value": "helpful"},
            {"type": "min_length", "value": 10},
        ],
    )
]
report = tester.test_content("You are a helpful assistant.", cases)
assert report.all_passed

CI/CD Integration

GitHub Actions

name: Prompt Quality
on: [push, pull_request]

jobs:
  lint-prompts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install promptkit
      - run: |
          promptkit init
          for f in prompts/*.md; do
            name=$(basename "$f" .md)
            promptkit save "$name" -f "$f"
            promptkit lint "$name"
          done

Configuration

.promptkit/config.json:

{
  "version": "1",
  "default_model": "gpt-4o",
  "default_provider": "openai",
  "prompt_format": "markdown",
  "template_engine": "jinja2",
  "pricing": {
    "gpt-4o": {"input": 2.50, "output": 10.00},
    "gpt-4o-mini": {"input": 0.15, "output": 0.60},
    "claude-sonnet-4-20250514": {"input": 3.00, "output": 15.00}
  }
}

Development

git clone https://github.com/bysiber/promptkit.git
cd promptkit
pip install -e ".[test]"
pytest

License

MIT License — see LICENSE for details.