Prompt Optimizer
Stop guessing. Start engineering. A CLI tool that scores your prompt against 10 principles and rewrites it into a production-ready, structured prompt — powered by Groq.
Architecture
principles.py ──▶ prompts.py
cli/ ──▶ analyzer.py / optimizer.py ──▶ llm.py ──▶ Groq API
│ │ │
│ └──▶ schemas.py └──▶ config.py
▼
storage.py
| Module | Role |
|---|---|
cli/ |
Click entry point, input gathering (arg / --file / stdin), pipeline orchestration |
cli/ui.py |
Rich console, colour helpers, and all report rendering |
cli/errors.py |
Turns pipeline failures into actionable terminal messages |
analyzer.py |
Step 1 — prompt + grading instructions → PromptAnalysis |
optimizer.py |
Step 2 — prompt + analysis + rewrite instructions → OptimizedPrompt |
principles.py |
The 10 rules as one data table |
prompts.py |
All LLM prompt text: both system prompts + human templates |
schemas.py |
Pydantic contracts: PromptWeakness, PromptAnalysis, OptimizedPrompt |
llm.py |
The only module that knows how to talk to Groq |
config.py |
Every setting in one place, each overridable by env var |
storage.py |
Builds a timestamped path per run and writes it |
run.sh |
Bootstraps the venv, checks the API key, then offers an interactive menu |
Dependencies point one way only: cli → analyzer/optimizer → llm → config.
principles and schemas sit at the bottom with no internal dependencies, so
either step can be swapped, retried, or tested without touching the other.
Workflow
User Prompt ──▶ Analyze ──▶ Quality Score (0-100)
│ + Strengths
│ + Weaknesses (with severity & suggestions)
▼
Optimize ──▶ Structured Prompt
+ Role / Objective / Context
+ Step-by-Step Instructions
+ Constraints & Rules
+ Output Format
+ Success Criteria
+ Changes Made Summary
+ Principles Applied List
The 10 Principles
Defined once in principles.py and reused everywhere:
- Role & Persona Definition — assign expertise level and perspective
- Clear Objective — one-sentence goal statement
- Context Provision — background, audience, environment
- Step-by-Step Instructions — sequential breakdown of complex tasks
- Constraints & Rules — boundaries, exclusions, length limits
- Output Format Specification — JSON, markdown, bullet points, etc.
- Few-Shot Examples — demonstrate desired pattern with examples
- Chain of Thought — request reasoning for multi-step problems
- Success Criteria — define measurable quality bar
- Specific Language — replace vague qualifiers with precise terms
Tech Stack
| Technology | Where Used | Purpose |
|---|---|---|
Groq (openai/gpt-oss-20b) |
src/prompt_optimizer_tool/llm.py |
Powers both LLM calls via ChatGroq. Free tier available |
LangChain (ChatGroq, ChatPromptTemplate) |
src/prompt_optimizer_tool/llm.py, analyzer.py, optimizer.py |
Provider abstraction, prompt templates, chain composition with LCEL |
Structured Outputs (with_structured_output) |
src/prompt_optimizer_tool/llm.py |
Groq's json_schema mode in strict mode uses constrained decoding, so every response matches the Pydantic schema — no JSON parse errors, and Literal fields can only return declared values |
Pydantic v2 (BaseModel, Field) |
src/prompt_optimizer_tool/schemas.py |
Data contracts for PromptWeakness, PromptAnalysis, OptimizedPrompt — validation + serialization |
Click (@click.group, @click.command, @click.option) |
src/prompt_optimizer_tool/cli/__init__.py |
CLI framework with composable commands, argument parsing, and auto-generated --help |
Rich (Console, Panel, Table, Markdown, Columns) |
src/prompt_optimizer_tool/cli/ui.py |
Terminal UI — color-coded score panels, weakness tables, markdown rendering, status spinners |
python-dotenv (load_dotenv) |
src/prompt_optimizer_tool/config.py |
Auto-loads GROQ_API_KEY from .env file — no manual config flags |
| pytest (offline suite) | tests/ |
80+ cases, no API key or network needed: CLI flow via CliRunner, the LLM seam with a fake provider, schemas, principles, config, storage, eval |
| uv | pyproject.toml |
Package manager — single uv sync installs deps, builds package, registers CLI entry point |
| Python 3.12 | .python-version, pyproject.toml |
Type hints (list[str], dict[str, str]), str.removeprefix/removesuffix |
Installation
Published on PyPI as prompt-optimizer-tool.
# Try it instantly, nothing is installed
uvx --from prompt-optimizer-tool prompt-optimizer list
# Install as an isolated tool (recommended)
uv tool install prompt-optimizer-tool
# or with pipx
pipx install prompt-optimizer-tool
Set your Groq API key
Get a key at https://console.groq.com/keys.
# Interactive setup — saves key permanently to ~/.prompt-optimizer/config.env
prompt-optimizer setup
# Or export it manually for the current session
export GROQ_API_KEY=gsk_your-key-here
Install from source (alternative)
git clone https://github.com/khadarbashajilan/prompt-optimizer-tool.git
cd prompt-optimizer-tool
cp .env.example .env # then edit .env: GROQ_API_KEY=gsk_your-key-here
./run.sh
Usage
# 1. First-time setup — save your Groq API key permanently
prompt-optimizer setup
# 2. Analyze + optimize a prompt
prompt-optimizer optimize "write code in python to fetch api data"
# Read from file
prompt-optimizer optimize --file prompt.txt
# Pipe from stdin
echo "explain quantum computing to a 5 year old" | prompt-optimizer optimize
# Each run is auto-saved to its own timestamped file in saved/
prompt-optimizer optimize "write a blog post"
# → saved/optimized-20260927-143205.md
# Choose a different output folder
prompt-optimizer optimize "write a blog post" --out ~/prompts
# List all 10 prompting principles
prompt-optimizer list
Configuration
Every setting has a default and can be overridden with an environment variable, so none of them require a code change:
| Variable | Default | Means |
|---|---|---|
GROQ_API_KEY |
(required) | Authorises the calls. Read from .env |
PROMPT_OPTIMIZER_MODEL |
openai/gpt-oss-20b |
Which Groq model to use |
PROMPT_OPTIMIZER_TEMPERATURE |
0.6 |
Higher means more varied wording |
PROMPT_OPTIMIZER_MAX_TOKENS |
8192 |
Reply ceiling; lower values truncate long rewrites |
PROMPT_OPTIMIZER_MAX_RETRIES |
3 |
Attempts per call, handles transient rate limits |
PROMPT_OPTIMIZER_TIMEOUT |
60 |
Per-request seconds, so a stalled call cannot hang |
PROMPT_OPTIMIZER_OUTPUT_DIR |
saved |
Default folder for saved rewrites |
Example Output
The tool produces a structured report:
- Quality Score — color-coded (green ≥70, yellow ≥40, red <40)
- Strengths — what the original prompt does well
- Weaknesses & Improvements — each issue with severity, violated principle, and concrete fix
- Changes Made — summary of all improvements applied
- Optimized Prompt — the complete rewritten prompt (copy-ready)
- Principles Applied — which of the 10 principles were used
- Auto-save — every run writes its own file to
saved/, named for the date and time of the run, so results accumulate instead of overwriting
Project Structure
prompt-optimizer/
pyproject.toml # uv project config with entry point
run.sh # bootstrap + interactive menu
CONTRIBUTING.md # setup, tests, and PR guidelines
.env.example # GROQ_API_KEY template
.gitignore
src/
prompt_optimizer_tool/
__init__.py # package version + layering notes
cli/
__init__.py # Click commands + pipeline orchestration
ui.py # Rich console, renderers, colour helpers
errors.py # user-facing error messages
__main__.py # python -m prompt_optimizer_tool.cli
analyzer.py # Step 1: grade a prompt
optimizer.py # Step 2: rewrite it
principles.py # 10 principles as data
prompts.py # system prompts + human templates
schemas.py # Pydantic contracts for structured I/O
llm.py # the only module that talks to Groq
config.py # all settings + API key loading
storage.py # timestamped path per run, then write
eval/
run_eval.py # stable entry point + facade
harness/ # internal harness modules
rubric.py # rubric constants + section detection
prompt_io.py # JSONL prompt loading
pipeline.py # Row + analyze/optimize/rescore execution
metrics.py # aggregation and headline metrics
report.py # markdown/JSON rendering + writing
data/
prompts.jsonl # 24 benchmark prompts
results/ # timestamped eval outputs
tests/
support.py # FakeProvider + fresh-model factories
conftest.py # fixtures for the two test seams
test_cli.py # end-to-end: input modes, exit codes, saving
test_pipeline.py # schema + message contracts at the LLM seam
test_principles.py # table/Literal/prompt consistency
test_schemas.py # validation bounds and Literal values
test_config.py # API key resolution + placeholder rejection
test_storage.py # timestamped naming, collision, folders
saved/ # one file per run, gitignored
media/
demo.gif # animated demo (auto-generated)
Evaluation Results
Benchmark run across 24 real-world prompts on 2026-11-08 (20 scored, 4 failed) — full metrics, scores, and principle coverage:
📄 eval/results/20261108-220810.md
Contributing
See CONTRIBUTING.md for setup, tests, code layout, and PR guidelines.
- Fork → branch → PR against
main - Run the offline test suite:
uv run pytest— CI enforces 94% coverage - No API key needed for tests; the LLM seam is faked via
FakeProviderintests/support.py - Run
prompt-optimizer setupto configure your Groq key for manual testing
License
Metadata
Release files for prompt-optimizer-tool 0.1.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| prompt_optimizer_tool-0.1.4.tar.gz | 30.5 kB | With provenance | Details for prompt_optimizer_tool-0.1.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| prompt_optimizer_tool-0.1.4-py3-none-any.whl | Python 3 | none | any | Details for prompt_optimizer_tool-0.1.4-py3-none-any.whl |
Total release size: 53.7 kB
Release files / prompt_optimizer_tool-0.1.4.tar.gz
| Download URL | prompt_optimizer_tool-0.1.4.tar.gz |
|---|---|
| Size | 30.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c3fa825daf86ffd88ce9f66db5c20f2776a8a48ec7038e66ac18d3765edbbfe9
|
|
BLAKE2b-256 checksum How to use checksums |
7b06646f6bb4d057a026b095460f080d3c56b45a86f4635c1dc6b88dc8bccbf8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.
Transparency logRelease files / prompt_optimizer_tool-0.1.4-py3-none-any.whl
| Download URL | prompt_optimizer_tool-0.1.4-py3-none-any.whl |
|---|---|
| Size | 23.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e9827707b9a6c1a4e8c5318c64e6e17a9164e74945735a31f824cbd0c6b76e75
|
|
BLAKE2b-256 checksum How to use checksums |
47418822fc6183d28c30c2648612fa615d4af03ca15b75930c3713da5c1c3163
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.
Transparency logSecurity for prompt-optimizer-tool 0.1.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for prompt_optimizer_tool-0.1.4.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.