AgentLab Python Client
A Python client library for the AgentLab evaluation platform using Connect RPC. This library provides a simple and intuitive interface for running AI agent evaluations, managing evaluators, and accessing evaluation results.
🚀 Quick Start
pip install agentlab-py
Set your API token as an environment variable:
export AGENTLAB_API_TOKEN=your-api-token-here
from agentlab import AgentLabClient, CreateEvaluationOptions
client = AgentLabClient()
evaluation = client.run_evaluation(CreateEvaluationOptions(
agent_name='my-agent',
agent_version='1.0.0',
evaluator_names=['correctness-v1'],
user_question='What is the capital of France?',
agent_answer='The capital of France is Paris.',
ground_truth='Paris is the capital of France',
metadata={'confidence': 0.95}
))
print(f"Evaluation completed: {evaluation.name}")
Retrieving Results
evaluation_run = client.get_evaluation_run('evaluation-run-id')
result_data = client.get_evaluation_result('evaluation-run-id')
print(result_data['results']) # Parsed evaluator outputs
for evaluator_name, result in evaluation_run.evaluator_results.items():
print(f"{evaluator_name}: {result.output}")
Listing Evaluation Runs
runs_response = client.list_evaluation_runs('project-123')
for run in runs_response.evaluation_runs:
print(f"Run: {run.name} - Question: {run.user_question}")
Managing Agent Prompts
from agentlab import CreateAgentVersionOptions
# Publish agent version with prompts (idempotent)
result = client.publish_agent_version(CreateAgentVersionOptions(
agent_name='my-assistant',
version='1.0.0',
prompts={
'system': 'You are a helpful AI assistant...',
'guidelines': 'Always be polite and professional.'
}
))
print(f"Published version: {result.create_time}")
for name, content in result.prompts.items():
print(f" {name}: {content[:50]}...")
Analyzing Agent Performance
from agentlab import AnalysisParameters
# Create analysis for the last 30 days
params = AnalysisParameters(min_evaluation_runs=5, time_range_days=30)
session = client.analyze_agent('my-agent', '1.0.0', params)
# Get results
session = client.get_analysis_session(session.id)
if session.status.value == "ANALYSIS_STATUS_COMPLETED":
stats = session.analysis_data.statistical_summary
print(f"Success rate: {stats.success_rate:.1%}")
print(f"Average score: {stats.average_score:.3f}")
Working with Test Datasets
# List all test datasets
datasets = client.list_test_datasets()
print(f"Found {len(datasets.test_datasets)} dataset(s)")
# Get tests from a dataset
tests = client.get_tests('dataset-uuid')
for test in tests.tests:
print(f"Q: {test.user_question}")
print(f"A: {test.ground_truth}")
🤝 Contributing
We welcome contributions! Please see our Contributing Guide for details.
Issues and Feature Requests
🔗 Links
🏢 About VectorLabs
AgentLab is developed by VectorLabs, a company focused on advancing AI agent evaluation and development tools.
Made with ❤️ by the VectorLabs team
Metadata
Release files for agentlab-py 0.6.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| agentlab_py-0.6.1.tar.gz | 86.0 kB | Without provenance | Details for agentlab_py-0.6.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| agentlab_py-0.6.1-py3-none-any.whl | Python 3 | none | any | Details for agentlab_py-0.6.1-py3-none-any.whl |
Total release size: 162.8 kB
Release files / agentlab_py-0.6.1.tar.gz
| Download URL | agentlab_py-0.6.1.tar.gz |
|---|---|
| Size | 86.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b4d826737a1ffb3be8c9830812bef9f356af795918881997f7127c56decb309a
|
|
BLAKE2b-256 checksum How to use checksums |
2a76868af6292a19f74005008fa811ec7aa1afbb53f37a00638ade8d99f159be
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.7
|
Release files / agentlab_py-0.6.1-py3-none-any.whl
| Download URL | agentlab_py-0.6.1-py3-none-any.whl |
|---|---|
| Size | 76.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2fafd424fa928c31889ac0b0c43ec448432b767b4d5487fcbe1caf0dcc97c753
|
|
BLAKE2b-256 checksum How to use checksums |
82e8bbd7e50aed356aab56531725ff8bcfad7fada9c384200169c6c069ed3298
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.7
|
Security for agentlab-py 0.6.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.