Rhesis SDK 🧠
Your team defines expectations, Rhesis generates and executes thousands of test scenarios. So that you know what you ship.
The Rhesis SDK empowers developers to programmatically access curated test sets and generate comprehensive test scenarios for Gen AI applications. Transform domain expertise into automated testing: access thousands of test scenarios, generate custom validation suites, and integrate seamlessly into your workflow to keep your Gen AI robust, reliable & compliant.
📑 Table of Contents
- Features
- Installation
- Getting Started
- Quick Start
- About Rhesis AI
- Community
- Hugging Face
- Support
- License
✨ Features
The Rhesis SDK provides programmatic access to the Rhesis testing platform:
- Access Test Sets: Browse and load curated test sets across multiple domains and use cases
- Generate Test Scenarios: Create custom test sets from prompts, requirements, or domain knowledge
- Seamless Integration: Integrate testing into your CI/CD pipeline and development workflow
- Comprehensive Coverage: Scale your testing from dozens to thousands of scenarios
- Open Source: MIT-licensed with full transparency and community-driven development
🚀 Installation
Install the Rhesis SDK using pip:
pip install rhesis-sdk
The core install connects your application to Rhesis: the connector, @endpoint, @observe and tracing, RhesisClient and the entities, @metric, and the native judges and synthesizers with the Rhesis-hosted model. For other model providers, DeepEval and DeepTeam metrics, document extraction and chunking, and MCP agents, install every SDK feature:
pip install "rhesis-sdk[all]"
Local Hugging Face models need the huggingface extra (rhesis-sdk[all,huggingface]). See What to install for every extra and for upgrading from 0.17.
🐍 Python Requirements
Rhesis SDK requires Python 3.12 or newer.
🏁 Getting Started
1. Obtain an API Key 🔑
- Visit https://app.rhesis.ai
- Sign up for a Rhesis account
- Navigate to your account settings
- Generate a new API key
Your API key will be in the format rh-XXXXXXXXXXXXXXXXXXXX. Keep this key secure and never share it publicly.
Note: On the Rhesis App, you can also create test sets for your own use cases and access them via the SDK. You only need to connect your GitHub account to create a test set.
2. Configure and use the SDK.
from pprint import pprint
from rhesis.sdk.entities import TestSet
from rhesis.sdk.synthesizers import PromptSynthesizer
os.environ["RHESIS_API_KEY"] = "rh-your-api-key" # Get from app.rhesis.ai settings
os.environ["RHESIS_BASE_URL"] = "https://api.rhesis.ai" # optional
# Browse available test sets
for test_set in TestSet().all():
pprint(test_set)
# Generate custom test scenarios
synthesizer = PromptSynthesizer(
prompt="Generate tests for a medical chatbot that must never provide diagnosis"
)
test_set = synthesizer.generate(num_tests=10)
pprint(test_set.tests)
Generating Custom Test Sets 🛠️
If none of the existing test sets fit your needs, you can generate your own. You can check out app.rhesis.ai. There you can define requirements, scenarios and expected behaviors.
Annotating Results 📝
An annotation records a person's verdict on a test result or a trace. A pass or fail annotation overrides the automated verdict.
from rhesis.sdk.entities import TestResults, TestRuns
from rhesis.sdk.telemetry import annotate_current_trace
# Read what people concluded about a run
run = TestRuns.pull(id="your-test-run-id")
for annotation in run.get_annotations():
print(annotation.status.name, annotation.user.name, annotation.comments)
# Overrule the automated verdict, for the whole result or one metric
result = TestResults.pull(id="your-test-result-id")
result.annotate("fail", "Cites a policy that does not exist.")
result.annotate("pass", "Relevant after all.", metric="Answer Relevancy")
# From inside an instrumented app, on the trace it is recording
annotate_current_trace("fail", "Answer failed our own validation check.")
See the annotations reference for traces, turns and querying.
🧪 About Rhesis AI
Rhesis is an open-source testing platform that transforms how Gen AI teams validate their applications. Through collaborative test management, domain expertise becomes comprehensive automated testing: legal defines requirements, marketing sets expectations, engineers build quality, and everyone knows exactly how the Gen AI application performs before users do.
Key capabilities:
- Collaborative Test Management: Your entire team contributes requirements without writing code
- Automated Test Generation: Generate thousands of test scenarios from team expertise
- Comprehensive Coverage: Scale from dozens of manual tests to thousands of automated scenarios
- Edge Case Discovery: Find potential failures before your users do
- Compliance Validation: Ensure systems meet regulatory and ethical standards
Made in Potsdam, Germany 🇩🇪
Visit rhesis.ai to learn more about our platform and services.
👥 Community 💬
Join our Discord server to connect with other users and developers.
🤗 Hugging Face
You can also find us on Hugging Face. There, you can find our test sets across multiple use cases.
🆘 Support
For questions, issues, or feature requests:
- Documentation: docs.rhesis.ai
- Discord Community: discord.rhesis.ai
- GitHub Discussions: Community discussions
- Email: hello@rhesis.ai
- Issues: Report bugs or request features
📝 License
The Rhesis SDK is licensed under the MIT License - see the LICENSE file for details.
The SDK is completely open-source and freely available for use, modification, and distribution.
Made with ❤️ in Potsdam, Germany 🇩🇪
Learn more at rhesis.ai
1 maintainer
Metadata
1 maintainer
Release files for rhesis-sdk 0.19.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| rhesis_sdk-0.19.0.tar.gz | 585.6 kB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| rhesis_sdk-0.19.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.3 MB
Release files / rhesis_sdk-0.19.0.tar.gz
| Download URL | rhesis_sdk-0.19.0.tar.gz |
|---|---|
| Size | 585.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b880842fe40b2c3e9b9770bec87eccb71d8009d9bca05b5eafc013447dbb9041
|
|
BLAKE2b-256 checksum How to use checksums |
e99d29a2ba910e28728035bba3440f83b8cccb95c577a0276f3aac345ce1946e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 21, 2026.
Transparency logRelease files / rhesis_sdk-0.19.0-py3-none-any.whl
| Download URL | rhesis_sdk-0.19.0-py3-none-any.whl |
|---|---|
| Size | 734.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c5482817877d87a7114081c43fd13a58467fbb5b1cee1078af101f6af74ed153
|
|
BLAKE2b-256 checksum How to use checksums |
067c1dbb831cf786c0c615e7826015042efc84e74bc6c2f4a61bb750e4c776ef
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 21, 2026.
Transparency logSecurity for rhesis-sdk 0.19.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for rhesis_sdk-0.19.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 21, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
