prompt-versioner 0.3.1

A comprehensive Python library for managing and versioning LLM prompts, with built-in A/B testing, metric tracking, and performance monitoring capabilities.

✨ Features • 🚀 Quick Start • 📖 Documentation • 🎯 Examples

🎯 Why Prompt Versioner?

Prompt Versioner provides enterprise-grade prompt management with:

• 🔄 Version Control: Full versioning of prompts
• 📊 Performance Tracking: Metrics and regression detection
• 🧪 A/B Testing: Built-in statistical framework
• ⚡ Real-time Monitoring: Alerts and dashboards
• 👥 Team Collaboration: Annotations and reviews
• 🎨 Modern UI: Responsive web dashboard

✨ Features

📦 Installation

Prerequisites

• Python 3.11+
• Poetry (recommended) or pip
• Git (optional)

Using PyPI

pip install prompt-versioner

🚀 Quick Start

from prompt_versioner import PromptVersioner, VersionBump

pv = PromptVersioner(project_name="my-first-project", enable_git=False)
pv.save_version(
    name="assistant-prompt",
    system_prompt="You are a helpful assistant.",
    user_prompt="Please answer the following question: {question}",
    bump_type=VersionBump.MAJOR
)

print("✅ Created first prompt version 1.0.0!")

🎨 Web Dashboard

Core Features

📋 Prompts Management

• Version history with visual diff
• Semantic search
• Bulk operations
• Live preview

📊 Metrics & Analytics

• Token usage, latency, cost analysis
• Quality score tracking
• Multi-model performance comparison with automatic "best model" detection

Model Comparison: Test the same prompt across different models (GPT-4, Claude, Gemini, etc.) and see aggregated metrics with automatic identification of the fastest, cheapest, and highest-quality model.

🧪 A/B Testing

• Split testing
• Real-time results

🔍 Version Comparison

• Side-by-side visual diff
• Metadata & performance delta
• Smart annotations

⚠️ Smart Alerts

• Automatic detection of regressions
• Cost and error monitoring
• Custom metrics & thresholds

💻 CLI Interface

# Initialization & Setup
pv init                              # Initialize prompt versioner

# Prompt Operations
pv list                              # List all tracked prompts
pv versions <prompt>                 # List versions of a prompt
pv show <prompt> <version>           # Show version details

# Model Pricing & Cost Estimation
pv models                            # List all models with pricing
pv models --sort-by input            # Sort by input price
pv models --filter gpt               # Filter specific models
pv estimate-cost <model> <in> <out>  # Estimate cost for usage
pv compare-costs <in> <out>          # Compare costs across models

# Version Comparison
pv diff <prompt> <v1> <v2>           # Show diff between versions
pv compare <prompt> <v1> <v2>        # Compare with metrics

# Management
pv delete <prompt> <version>         # Delete a version
pv rollback <prompt> <version>       # Rollback to version
pv clear-db                          # Clear database

# Dashboard
pv dashboard --port 5000             # Launch web dashboard

📖 Examples

Examples in examples/ are fully functional:

🌟 Contributing

See CONTRIBUTING.md for contribution guidelines.

📄 License

MIT License - LICENSE

📞 Support

• Documentation: https://pepes97.github.io/prompt-versioner/
• Issues: GitHub Issues
• Discussions: GitHub Discussions

Build by Sveva Pepe, NLP Engineer
_{⭐ Star this project if it helps you build better AI applications!}