scorable 1.7.2

The Python SDK for API of Scorable

Measurement & Control for LLM Automations

Scorable streamlines the evaluation of your LLM and agentic pipelines. We provide a holistic approach to GenAI measurability & observability with carefully-crafted ready-to-use evaluators based on cutting-edge LLM research as well as a framework for systematically adding your own custom evaluators.

With Scorable you can develop your LLM application reliably, deploy them in confidence, and ensure optimal performance with continuous monitoring.

📦 Install

pip install scorable

⚡ Quickstart

🔑 Get Your API Key

Sign up & create a key or generate a temporary key

Setup Option 1: Environment Variable

export SCORABLE_API_KEY=your-scorable-api-key

Setup Option 2: .env File

echo SCORABLE_API_KEY=your-scorable-api-key >> .env

Root Evaluators

from scorable import Scorable

# Connect to Scorable API
client = Scorable()

# Run any of our ready-made evaluators
result = client.evaluators.Politeness(
    response="You can find the instructions from our Careers page."
)
# Example result:
#   {
#     "score": 0.7, --> a normalized score between [0, 1]
#     "justification": "The response is st...",
#     "execution_log_id": "..."
#   }

Check the full list of Root evaluators from the Root evaluators documentation. You can also add your own custom evaluators.

📖 Documentation

Resource	Link
🐍 Python SDK	View Documentation
📘 Product Docs	View Documentation
📑 API Docs	View Documentation
🔌 MCP	View Repo

Tutorial	Link
Quickstart
Tuning a Custom Judge