promptsim 0.1.0

Open source SDK for prompt versioning, A/B testing, and cost tracking

PromptSim Core

Open source SDK for prompt versioning, A/B testing, and cost tracking for ChatGPT applications.

Features

• 🔄 Prompt Versioning: Git-like versioning for your prompts
• 🧪 A/B Testing: Simple experimentation with statistical significance
• 💰 Cost Tracking: Automatic OpenAI cost calculation and monitoring
• 📊 Local Storage: SQLite-based local storage with data portability
• ☁️ Cloud Ready: Seamless upgrade to PromptSim Cloud
• 🔒 Zero Lock-in: Full data export/import capabilities

Quick Start

pip install promptsim

from promptsim import PromptSim

# Local mode (no external dependencies)
client = PromptSim(storage="local")

# Create a prompt
client.create_prompt(
    key="welcome_message",
    template="Hello {name}! Welcome to our {product}.",
    model="gpt-3.5-turbo"
)

# Use the prompt
response = client.complete(
    prompt_key="welcome_message",
    variables={"name": "Alice", "product": "ChatGPT app"},
    user_id="user_123"
)

print(response.text)  # "Hello Alice! Welcome to our ChatGPT app."
print(response.cost_usd)  # 0.0001

Testing Your Installation

After installing PromptSim, you can verify everything works correctly:

Quick Test

# Test basic functionality (no OpenAI API key required)
from promptsim import PromptSim

client = PromptSim(storage="local")
prompt = client.create_prompt(
    key="test",
    template="Hello {name}!"
)
print(f"Created prompt: {prompt.key} v{prompt.version}")

Comprehensive Test Suite

# Run the built-in test runner
python -c "import promptsim; exec(open(promptsim.__file__.replace('__init__.py', '../run_tests.py')).read())"

# Or if you have the source code:
python run_tests.py

Manual Testing Steps

1. Test Basic Functionality (no API key needed):

from promptsim import PromptSim

# Create client
client = PromptSim(storage="local", db_path="test.db")

# Create and version prompts
v1 = client.create_prompt(key="test", template="Hello {name}!")
v2 = client.create_prompt(key="test", template="Hi {name}!")

# List prompts
prompts = client.list_prompts()
print(f"Created {len(prompts)} prompt versions")

# Export data (for backup/migration)
data = client.export_data()
print(f"Exported {len(data['prompts'])} prompts")

2. Test with OpenAI (requires API key):

import os
os.environ["OPENAI_API_KEY"] = "your-key-here"

from promptsim import PromptSim

client = PromptSim(storage="local")
client.create_prompt(
    key="welcome",
    template="Welcome {name} to {product}!"
)

response = client.complete(
    prompt_key="welcome",
    variables={"name": "Alice", "product": "PromptSim"},
    user_id="user123"
)

print(f"Response: {response.text}")
print(f"Cost: ${response.cost_usd:.6f}")

3. Test CLI Interface:

# Create a prompt
python -m promptsim.cli create "test_cli" "Hello {name}!" --name "CLI Test"

# List prompts
python -m promptsim.cli list --verbose

# Export data
python -m promptsim.cli export --output backup.json

Running Unit Tests

If you want to run the full test suite:

# Install test dependencies
pip install promptsim[test]

# Run tests
pytest tests/ -v

Troubleshooting

Import Error: Make sure you've installed all dependencies:

pip install openai tiktoken sqlalchemy requests pydantic

OpenAI API Error: The core functionality works without OpenAI. Set your API key only when you need to make actual completions:

export OPENAI_API_KEY="your-openai-api-key"

Permission Error: If you get database permission errors, make sure you have write access to the current directory or specify a different path:

client = PromptSim(storage="local", db_path="/tmp/promptsim.db")

Installation

From PyPI (Recommended)

pip install promptsim

From TestPyPI (Latest Development)

pip install -i https://test.pypi.org/simple/ promptsim

Local Development

git clone https://github.com/promptsim/promptsim-core
cd promptsim-core
pip install -e .

Storage Options

Local SQLite (Default)

client = PromptSim(storage="local", db_path="./prompts.db")

Cloud API

client = PromptSim(
    storage="cloud",
    api_key="pk_live_...",
    host="api.promptsim.com"
)

Local Files (JSON)

client = PromptSim(storage="files", data_dir="./prompt_data")

License

MIT License - see LICENSE file.

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.