Skip to main content

🧬 PromptAnalyzer

Git for prompts — local-first LLM observability & prompt versioning

One decorator. Zero config. No Docker, no cloud, no npm.

CI PyPI Python License: MIT Ruff


PromptAnalyzer gives any LLM-powered Python function automatic prompt versioning, inference logging, token & cost tracking, and a local dashboard — by adding a single @track decorator. Everything runs on your machine against SQLite. Nothing leaves your laptop.

from promptanalyzer import track


@track("medical-chatbot")
def ask(message):
    return client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a doctor's assistant."},
            {"role": "user", "content": message},
        ],
    )
promptanalyzer dashboard   # → http://localhost:4001

✨ Features

🔖 Prompt versioning Every system prompt is SHA-256 hashed and versioned automatically — like Git commits for prompts.
🧾 Full inference logs User input, system prompt, response, model, provider, timing — all captured.
⚡ Zero overhead Sub-millisecond hot path; writes happen on a background thread.
💸 Token & cost tracking Built-in pricing for OpenAI, Anthropic, Google, Mistral, Groq and more.
🔌 Provider agnostic Auto-detects OpenAI, Claude, Gemini, Ollama, vLLM, LiteLLM, OpenRouter, Groq, Mistral, Azure — or bring your own.
🪝 Auto-instrumentation Captures the real request & response from the SDK call inside your function — even if you only return the answer string.
📊 Local dashboard Server-side rendered (FastAPI + HTMX + Alpine). No React, no build step.
🔍 Search & diff Full-text search across prompts/responses and GitHub-style version diffs.
🛟 Never crashes your app Logging failures are swallowed and logged — your application keeps running.

🚀 Quick start

pip install "promptanalyzer[dashboard]"
  1. Decorate an LLM function with @track("project-name").
  2. Run your application as usual.
  3. Open the dashboard:
promptanalyzer dashboard
# http://localhost:4001

That's it. PromptAnalyzer creates ~/.promptanalyzer/promptanalyzer.db on first use.

Advanced decorator

@track(
    name="medical-assistant",
    tags=["production"],
    metadata={"team": "AI"},
)
def chatbot(message): ...

Any library (generic adapter)

@track(
    name="custom-model",
    system=lambda args, kwargs: kwargs["system"],
    user=lambda args, kwargs: kwargs["prompt"],
    response=lambda result: result,
)
def my_llm(system, prompt): ...

🖥️ Dashboard

Overview

Diff viewer

Overview Prompt versions Diff viewer Run detail
Totals + runs/tokens/cost/latency charts Every version with per-version metrics GitHub-style added/removed lines System prompt, messages, timing, tokens, cost

The images above are placeholders. To capture real screenshots, follow docs/screenshots.md.

📚 Documentation

Full guides live in docs/: Quickstart · Configuration · Providers & adapters · Dashboard · CLI · Prompt versioning · Database & migrations · Performance · FAQ.

🔌 Supported providers

OpenAI · Anthropic Claude · Google Gemini · Ollama · vLLM · LiteLLM · OpenRouter · Groq · Mistral · Azure OpenAI · any custom library via the generic adapter.

See examples/ for a runnable script per provider.

⚙️ Configuration

Zero config by default. Override via environment variables:

PROMPTANALYZER_DB=sqlite
PROMPTANALYZER_SQLITE_PATH=~/.promptanalyzer/promptanalyzer.db
PROMPTANALYZER_DATABASE_URL=postgresql://user:password@localhost/dbname

PROMPTANALYZER_HOST=127.0.0.1
PROMPTANALYZER_PORT=4001

PROMPTANALYZER_AUTO_START=true      # start dashboard on import
PROMPTANALYZER_OPEN_BROWSER=true

PROMPTANALYZER_LOG_TOKENS=true
PROMPTANALYZER_LOG_COST=true
PROMPTANALYZER_SAVE_RESPONSES=true

PROMPTANALYZER_PROJECT=default
PROMPTANALYZER_ENV=development

Priority: decorator arguments → environment variables → defaults. Legacy PROMPTLOG_* variables are also accepted.

🛠️ CLI

promptanalyzer init        # create ~/.promptanalyzer and the database
promptanalyzer dashboard   # launch the dashboard
promptanalyzer migrate     # create/upgrade the schema
promptanalyzer export csv  # export runs (json | csv | markdown)
promptanalyzer doctor      # diagnose your installation
promptanalyzer reset       # wipe all local data

🧱 Architecture

@track → adapter (normalizes any provider) → background writer → SQLite → FastAPI dashboard. See ARCHITECTURE.md for the full design.

🗺️ Roadmap

  • OpenTelemetry export
  • Prompt evaluation & A/B testing
  • Dataset management & prompt playground
  • Cloud sync + team collaboration
  • Authentication & multi-user
  • Plugin marketplace

🤝 Contributing

Contributions welcome! See CONTRIBUTING.md.

📄 License

MIT © PromptAnalyzer Contributors

Release files for promptanalyzer 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for promptanalyzer 0.1.0
File Size Uploaded
promptanalyzer-0.1.0.tar.gz 48.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for promptanalyzer 0.1.0
File Interpreter ABI Platform
promptanalyzer-0.1.0-py3-none-any.whl Python 3 none any Details

Total release size: 107.3 kB

Release files / promptanalyzer-0.1.0.tar.gz

Download URL promptanalyzer-0.1.0.tar.gz
Size 48.2 kB
Tags Source
SHA-256 checksum
How to use checksums
9355a836644157417897fdaacebe5344e27fb299b5778db2355d64539d040e71
BLAKE2b-256 checksum
How to use checksums
b9973479fe59b46a87ff684b20d06042160a320b646fcbb9c88eca5fd0e32d31
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release files / promptanalyzer-0.1.0-py3-none-any.whl

Download URL promptanalyzer-0.1.0-py3-none-any.whl
Size 59.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
72dda8a45a85afd182e70763183c7daf34b8883d336a3d0c66d256a2a0b32bee
BLAKE2b-256 checksum
How to use checksums
46eb3e3935f8006d82c1dd2aa0756f30f3038dd57d02aa85ef2f0e0c221180d0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release history Release notifications | RSS feed

0.1.1

2 release files

This release

0.1.0 This release

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page