tumeryk-guardrails 0.3.1

API Client for Tumeryk Guardrails - OpenAI-compatible interface with AI safety guardrails

Tumeryk Guardrails Client

The Tumeryk Guardrails Client is a Python package that provides an interface to the Tumeryk Guardrails API. The client allows you to easily use the API from your Python code.

Setup

To install the Tumeryk Guardrails Client, use pip:

pip install tumeryk_guardrails

Authentication Methods

Method	Endpoint	Client
API Key (tmryk_xxx)	/openai/v1/chat/completions	TumerykOpenAI or chat_completions()
Username/Password (JWT)	/v1/chat/completions	tumeryk_completions()

Example .env File

TUMERYK_USERNAME=sample_username
TUMERYK_PASSWORD=sample_password
TUMERYK_POLICY=hr_policy
TUMERYK_API_KEY=tmryk_your_api_key_here
TUMERYK_BASE_URL=https://chat.tmryk.com

---

OpenAI-Compatible API (API Key Authentication)

Quick Start with TumerykOpenAI

from tumeryk_guardrails import TumerykOpenAI

# Initialize with API key
client = TumerykOpenAI(
    api_key="tmryk_your_api_key_here",
    base_url="https://chat.tmryk.com"  # or your Guard server URL
)

# Use exactly like OpenAI's client
response = client.chat.completions.create(
    model="your_policy_name",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

# Access Tumeryk-specific metrics
print(f"Trust Score: {response.metrics.get('trust_score')}")
print(f"Model Score: {response.metrics.get('model_score')}")

Using Environment Variables

from tumeryk_guardrails import TumerykOpenAI

# API key is read from TUMERYK_API_KEY or OPENAI_API_KEY env var
client = TumerykOpenAI()

response = client.chat.completions.create(
    model="my_policy",
    messages=[{"role": "user", "content": "Hello!"}]
)

Using the Default Client with API Key

import tumeryk_guardrails

# Set API key
tumeryk_guardrails.set_api_key("tmryk_your_api_key_here")
tumeryk_guardrails.set_base_url("https://chat.tmryk.com")

# Use OpenAI-compatible endpoint
response = tumeryk_guardrails.chat_completions(
    messages=[{"role": "user", "content": "Hello!"}],
    model="your_policy_name"
)

print(response)

Drop-in OpenAI Replacement

TumerykOpenAI is designed as a drop-in replacement for the OpenAI Python client:

# Before (OpenAI)
from openai import OpenAI
client = OpenAI(api_key="sk-...")
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)

# After (Tumeryk with guardrails)
from tumeryk_guardrails import TumerykOpenAI
client = TumerykOpenAI(api_key="tmryk_...")
response = client.chat.completions.create(
    model="your_policy_name",
    messages=[{"role": "user", "content": "Hello!"}]
)

Function Calling / Tools

from tumeryk_guardrails import TumerykOpenAI

client = TumerykOpenAI(api_key="tmryk_...")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"}
                },
                "required": ["location"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="your_policy_name",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
    tool_choice="auto"
)

# Handle tool calls
if response.choices[0].message.tool_calls:
    for tool_call in response.choices[0].message.tool_calls:
        print(f"Function: {tool_call['function']['name']}")
        print(f"Arguments: {tool_call['function']['arguments']}")

Streaming Responses

from tumeryk_guardrails import TumerykOpenAI

client = TumerykOpenAI(api_key="tmryk_...")

# Stream the response
for chunk in client.chat.completions.create(
    model="your_policy_name",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
):
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

All Supported Parameters

response = client.chat.completions.create(
    model="your_policy_name",
    messages=[{"role": "user", "content": "Hello!"}],
    temperature=0.7,
    top_p=0.9,
    max_tokens=500,
    frequency_penalty=0.0,
    presence_penalty=0.0,
    stop=["\n"],
    user="user-123",
    stream=False,
    tools=None,
    tool_choice=None,
    response_format={"type": "text"}
)

Agent Tracking

Track different agents/workflows using the model namespace format:

# Direct policy reference
response = client.chat.completions.create(
    model="hr_policy",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Explicit policy reference with namespace
response = client.chat.completions.create(
    model="policy:hr_policy",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Agent tracking - looks up agent role and enables run tracking
response = client.chat.completions.create(
    model="agent:research_agent",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Use metadata for additional session tracking
response = client.chat.completions.create(
    model="agent:my_agent",
    messages=[{"role": "user", "content": "Hello!"}],
    metadata={
        "session_id": "user-session-123",
        "workflow_id": "onboarding-flow"
    }
)

Model Namespace Formats

Format	Description
policy_name	Direct config/policy reference
policy:policy_name	Explicit policy reference
agent:agent_name	Agent role lookup with run tracking

OpenAI-Compatible Response Structure

response = client.chat.completions.create(...)

# Standard OpenAI fields
print(response.id)
print(response.model)
print(response.choices[0].message.content)
print(response.usage.total_tokens)

# Tumeryk metrics
metrics = response.metrics
print(f"Trust Score: {metrics.get('trust_score')}")
print(f"Model Score: {metrics.get('model_score')}")
print(f"Real-time Score: {metrics.get('real_time_score')}")
print(f"Jailbreak Score: {metrics.get('jailbreak_score')}")
print(f"Violation: {metrics.get('violation')}")
print(f"Topic Relevance: {metrics.get('topic_relevance')}")

---

Tumeryk API (Username/Password Authentication)

Simple Usage

You can use the Tumeryk Guardrails Client with minimal setup. The client will automatically load the configuration from the .env file if it exists. Here's an example of simple usage:

from dotenv import load_dotenv
load_dotenv()

import tumeryk_guardrails

messages = [{"role": "user", "content": "hi"}]

response = tumeryk_guardrails.tumeryk_completions(messages=messages)

print(response)

Manual Usage

The Tumeryk Guardrails Client uses chat.tmryk.com as the default base URL. However, you can change this URL if required. Here's how you can set a custom base URL:

import tumeryk_guardrails

# Set a custom base URL
tumeryk_guardrails.set_base_url("https://your-custom-url.com")

Configuration

The Tumeryk Guardrails Client uses environment variables to store credentials and policy. Here's an example of how to use environment variables:

import tumeryk_guardrails
import os
from dotenv import load_dotenv

# Load environment variables
load_dotenv()

# Retrieve credentials and policy from environment variables
username = os.getenv("TUMERYK_USERNAME")
password = os.getenv("TUMERYK_PASSWORD")
policy = os.getenv("TUMERYK_POLICY")

# Authenticate with Tumeryk Guardrails
tumeryk_guardrails.login(username, password)

# Retrieve available policies
policies = tumeryk_guardrails.get_policies()
print("Available Policies:", policies)

# Set the chosen policy
tumeryk_guardrails.set_policy(policy)

# Prepare a message for the guard service
messages = [{"role": "user", "content": "Example input to guard"}]

# Send a request to the guard service
response = tumeryk_guardrails.tumeryk_completions(messages)
print("Guard Response:")
print(response)

Async Usage

For non-blocking requests, you can use the async version of the completions method. This is useful when you need to make multiple requests concurrently or want to avoid blocking your main thread:

import asyncio
import tumeryk_guardrails

async def async_example():
    # Authenticate (this can be done once)
    tumeryk_guardrails.login(username, password)
    tumeryk_guardrails.set_policy(policy)
    
    # Prepare messages
    messages = [{"role": "user", "content": "Example async input"}]
    
    # Send async request
    response = await tumeryk_guardrails.tumeryk_completions_async(messages)
    print("Async Guard Response:")
    print(response)

# Run the async function
asyncio.run(async_example())

Multiple Concurrent Requests

You can also make multiple requests concurrently:

import asyncio
import tumeryk_guardrails

async def multiple_requests():
    # Setup authentication and policy
    tumeryk_guardrails.login(username, password)
    tumeryk_guardrails.set_policy(policy)
    
    # Prepare multiple messages
    messages_list = [
        [{"role": "user", "content": "First request"}],
        [{"role": "user", "content": "Second request"}],
        [{"role": "user", "content": "Third request"}]
    ]
    
    # Make concurrent requests
    tasks = [
        tumeryk_guardrails.tumeryk_completions_async(messages) 
        for messages in messages_list
    ]
    
    responses = await asyncio.gather(*tasks)
    
    for i, response in enumerate(responses):
        print(f"Response {i+1}:", response)

# Run concurrent requests
asyncio.run(multiple_requests())

---

Advanced Features

Setting a Default Model Score

You can manually set a default model score that will be used for all completion requests. This is useful when you want to override the automatic model scoring:

import tumeryk_guardrails

# Set a default model score for all requests
tumeryk_guardrails.set_model_score(737)

# All subsequent completion calls will use this score
messages = [{"role": "user", "content": "Your prompt here"}]
response = tumeryk_guardrails.tumeryk_completions(messages)

You can also override the model score for individual requests:

import tumeryk_guardrails

# Set a default model score
tumeryk_guardrails.set_model_score(737)

messages = [{"role": "user", "content": "Your prompt here"}]

# Override the default score for this specific request
response = tumeryk_guardrails.tumeryk_completions(
    messages,
    manual_model_score=850
)

Controlling Rails Execution with Generation Options

The generation_options parameter allows you to control which security rails (input, output, dialog, retrieval) are executed during the guardrail check. This provides fine-grained control over the validation process:

import tumeryk_guardrails

# Example: Run only output and dialog rails, skip input and retrieval
generation_options = {
    "rails": {
        "input": False,    # Skip input validation
        "output": True,    # Run output validation
        "dialog": True,    # Run dialog validation
        "retrieval": False # Skip retrieval validation
    }
}

messages = [
    {"role": "user", "content": "What is the capital of France?"},
    {"role": "assistant", "content": "The capital of France is Paris."}
]

response = tumeryk_guardrails.tumeryk_completions(
    messages,
    generation_options=generation_options
)

Complete Example with All Advanced Features

Here's a comprehensive example demonstrating all advanced features together:

import asyncio
import tumeryk_guardrails
from dotenv import load_dotenv
import os

load_dotenv()

async def advanced_example():
    # Authenticate
    username = os.getenv("TUMERYK_USERNAME")
    password = os.getenv("TUMERYK_PASSWORD")

    tumeryk_guardrails.login(username, password)

    # Set a default model score
    tumeryk_guardrails.set_model_score(737)

    # Define the input prompt
    user_prompt = "What is the capital of Mozambique?"
    input_messages = [{"role": "user", "content": user_prompt}]

    # Run input policy check
    input_response = await tumeryk_guardrails.tumeryk_completions_async(
        input_messages,
        policy_id="your_policy_name"
    )

    print("Input Check Response:", input_response)

    # Simulate LLM response
    llm_response = "The capital of Mozambique is Maputo."

    # Check output with specific rails configuration
    output_messages = [
        {"role": "user", "content": user_prompt},
        {"role": "assistant", "content": llm_response}
    ]

    # Configure to run only output and dialog rails
    output_rails_config = {
        "rails": {
            "input": False,
            "output": True,
            "dialog": True,
            "retrieval": False
        }
    }

    # Run output policy check with custom configuration
    output_response = await tumeryk_guardrails.tumeryk_completions_async(
        output_messages,
        policy_id="your_policy_name",
        generation_options=output_rails_config,
        manual_model_score=800  # Override default score for this call
    )

    print("Output Check Response:", output_response)

# Run the example
asyncio.run(advanced_example())

---

Response Structure

The response format differs between the two endpoints:

TumerykOpenAI Response (OpenAI-Compatible)

{
    "id": "chatcmpl-abc123",
    "object": "chat.completion",
    "created": 1234567890,
    "model": "your_policy_name",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "The capital of France is Paris."
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 15,
        "completion_tokens": 10,
        "total_tokens": 25
    },
    "metrics": {
        "trust_score": 850,
        "model_score": 800,
        "real_time_score": 825,
        "input": {
            "fairness_score": 990,
            "moderation_score": 1000,
            "moderation_violations": "Safe",
            "topic_relevance": 1000,
            "jailbreak_score": 995
        },
        "output": {
            "fairness_score": 985,
            "moderation_score": 1000,
            "moderation_violations": "Safe",
            "hallucination_score": -1
        },
        "tools": {
            "requested": 0,
            "executed": 0
        }
    }
}

Accessing TumerykOpenAI response:

response = client.chat.completions.create(...)

# Standard OpenAI fields
print(response.choices[0].message.content)
print(response.usage.total_tokens)

# Tumeryk metrics
print(response.metrics["trust_score"])
print(response.metrics["input"]["jailbreak_score"])
print(response.metrics["output"]["hallucination_score"])

tumeryk_completions Response (Tumeryk API)

{
    "messages": [
        {
            "role": "assistant",
            "content": "The capital of France is Paris.",
            "stats": {
                "total_calls": 1,
                "total_time": 2.5,
                "total_tokens": 25,
                "total_prompt_tokens": 15,
                "total_completion_tokens": 10
            }
        }
    ],
    "metrics": {
        "violation": false,
        "jailbreak_detection": false,
        "topic_relevance": 1000,
        "trust_score": 850,
        "model_score": 800,
        "real_time_score": 825,
        "hallucination_score": -1.0,
        "bias_score": {
            "input": 990,
            "output": 985
        },
        "toxicity_scores": {
            "input": "Safe",
            "output": "Safe"
        },
        "llama_guard_allowed": {
            "input": true,
            "output": true
        },
        "llama_guard_categories": {
            "input": "Safe",
            "output": "Safe"
        },
        "information": {
            "300": "Low Prompt Injection Score"
        },
        "jailbreak_score": 995,
        "moderation_score_input": 1000,
        "moderation_score_output": 1000
    },
    "log": "# General stats\n- Total time: 2.5s\n..."
}

Accessing tumeryk_completions response:

response = tumeryk_guardrails.tumeryk_completions(messages)

# Message content
print(response["messages"][0]["content"])

# Metrics
print(response["metrics"]["trust_score"])
print(response["metrics"]["jailbreak_score"])
print(response["metrics"]["bias_score"]["input"])

Metrics Field Reference

Metric	TumerykOpenAI Location	tumeryk_completions Location	Description
Trust Score	metrics.trust_score	metrics.trust_score	Overall trust rating (0-1000)
Model Score	metrics.model_score	metrics.model_score	Model quality score (0-1000)
Real-time Score	metrics.real_time_score	metrics.real_time_score	Real-time processing score
Jailbreak Score	metrics.input.jailbreak_score	metrics.jailbreak_score	Protection against jailbreak (0-1000)
Bias (Input)	metrics.input.fairness_score	metrics.bias_score.input	Input fairness score
Bias (Output)	metrics.output.fairness_score	metrics.bias_score.output	Output fairness score
Hallucination	metrics.output.hallucination_score	metrics.hallucination_score	Hallucination detection (-1 if not run)
Topic Relevance	metrics.input.topic_relevance	metrics.topic_relevance	Topic relevance score
Moderation (Input)	metrics.input.moderation_score	metrics.moderation_score_input	Input safety score
Moderation (Output)	metrics.output.moderation_score	metrics.moderation_score_output	Output safety score
Violation	N/A	metrics.violation	Boolean policy violation flag

Log (tumeryk_completions only): Detailed processing statistics showing:

• Processing time breakdown by rail type (INPUT, DIALOG, OUTPUT)
• LLM call statistics and token usage
• Individual action execution times and details
• Performance metrics for each processing stage

---

Available Methods

OpenAI-Compatible (API Key Auth)

Method	Description
TumerykOpenAI(api_key, base_url)	Create OpenAI-compatible client
client.chat.completions.create(...)	Create chat completion
set_api_key(key)	Set API key for authentication
chat_completions(messages, model, ...)	OpenAI-compatible completions
chat_completions_async(messages, model, ...)	Async OpenAI-compatible completions

Tumeryk API (Username/Password Auth)

The Tumeryk Guardrails Client provides the following methods to interact with the Tumeryk Guardrails API:

• login(username, password): Authenticate and store access token.
• set_token(token): Set a token directly without authentication.
• get_policies(): Fetch available policies and return a list.
• set_policy(config_id): Set the configuration/policy to be used by the user.
• set_model_score(score): Set a default manual model score for all completion calls.
• tumeryk_completions(messages): Send user input to the Guard service.
• tumeryk_completions_async(messages): Async version of tumeryk_completions for non-blocking requests.
• get_base_url(): Get the current base URL.
• set_base_url(base_url): Set a new base URL.

Detailed Parameters for Completion Methods

Both tumeryk_completions and tumeryk_completions_async support the following parameters:

• messages (required): List of message dictionaries with 'role' and 'content' keys.
• stream (optional): Boolean to enable streaming responses (default: False).
• policy_id (optional): Override the default policy for this specific request.
• generation_options (optional): Dictionary to control which rails to execute. Format:

  {
      "rails": {
          "input": True/False,
          "output": True/False,
          "dialog": True/False,
          "retrieval": True/False
      }
  }
  

• manual_model_score (optional): Integer to override the model score for this specific request.

---

Migration Guide

From Username/Password to API Key

# Before (username/password)
import tumeryk_guardrails
tumeryk_guardrails.login("user", "pass")
tumeryk_guardrails.set_policy("my_policy")
response = tumeryk_guardrails.tumeryk_completions(messages)

# After (API key with TumerykOpenAI)
from tumeryk_guardrails import TumerykOpenAI
client = TumerykOpenAI(api_key="tmryk_...")
response = client.chat.completions.create(
    model="my_policy",
    messages=messages
)

From OpenAI SDK

# Before
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}]
)

# After - just change the import and client initialization
from tumeryk_guardrails import TumerykOpenAI
client = TumerykOpenAI(api_key="tmryk_...")
response = client.chat.completions.create(
    model="your_policy",  # Use your Tumeryk policy name
    messages=[{"role": "user", "content": "Hello"}]
)
# Same response format, with additional .metrics field

---

Dependencies

• requests: Used for making HTTP requests to the API.
• aiohttp: Used for making async HTTP requests to the API.

Support

For support, contact [e-mail] or visit https://tumeryk.com.