SENTINEL Python SDK (sentinel-eval-sdk v3.0.2)

Official Python Client for SENTINEL: Autonomous LLMOps with Hybrid Subword-Dense Embedding Cosine Similarity, Real-Time Faithfulness Verification, and Closed-Loop Prompt Self-Healing.

---

📩 Reviewer Contact & Peer Review Feedback

For academic paper feedback, research review, bug reports, or enterprise integration inquiries, please reach out directly:

• 👤 Author & Lead Developer: Srishanth Reddy
• 📧 Review / Feedback Email: srishanthreddy311030@gmail.com
• 🌐 ResearchGate Publication: SENTINEL Autonomous LLMOps (Publication #414271476)
• 📦 PyPI Package: https://pypi.org/project/sentinel-eval-sdk/
• 🐙 GitHub Repository: https://github.com/srishanthreddy456789/Sentinel

---

📌 Executive Summary

SENTINEL (sentinel-eval-sdk) inserts a continuous, sub-20ms AI Quality Verification Layer into production LLM and RAG pipelines. Operating completely local-first without external cloud API dependencies, SENTINEL tracks predictions, detects hallucinations, verifies context entailment, evaluates 9 metric dimensions, diagnoses failure root causes, and automatically repairs prompt regressions using closed-loop self-healing algorithms.

---

🚀 Installation

Install the official package from PyPI:

pip install sentinel-eval-sdk

---

⚡ 3-Line Decorator Quickstart

Monitor any LLM execution function or RAG chain with zero latency overhead ($<0.1\text{ ms}$ main-thread queuing):

import sentinel_sdk as sentinel

# 1. Initialize SENTINEL SDK with backend URL & API Key
sentinel.init(api_key="sk_sentinel_2026", base_url="http://localhost:8000")

# 2. Wrap your LLM invocation function
@sentinel.monitor(model_id="customer_support_bot")
def generate_response(user_query: str):
    # Call your local Ollama instance, LangChain RAG pipeline, or custom model
    return "Refunds are processed within 30 days of purchase."

# 3. Execute function normally — telemetry is dispatched asynchronously in background!
response = generate_response("What is the refund policy?")
print("Response:", response)

---

🧰 Detailed Function Reference & Technical Guide

The SDK exports 7 predefined feature functions matching every tab in the SENTINEL Desktop App interface.

---

1. sentinel.playground() — Interactive Execution & Metric Testing

Desktop App Equivalent: 🧪 Playground Tab

Description

Executes interactive model inputs against reference targets or retrieved context, performing real-time 9-metric evaluation. Used during prompt development, rapid testing, and regression checking.

Function Signature

def playground(
    input_text: str,
    expected_output: Optional[str] = None,
    context: Optional[str] = None,
    model_id: str = "default_model"
) -> Dict[str, Any]:

Parameters

Parameter	Type	Required	Default	Description
input_text	str	Yes	—	The user prompt or evaluation query string.
expected_output	str	No	None	Ground-truth reference text for cosine similarity evaluation.
context	str	No	None	Retrieved document context chunks for RAG faithfulness verification.
model_id	str	No	"default_model"	Model identifier to tag evaluation records in the dashboard.

Code Example

import sentinel_sdk as sentinel

metrics = sentinel.playground(
    input_text="What is the guaranteed SLA uptime?",
    expected_output="SENTINEL guarantees 99.9% uptime SLA.",
    context="Official Policy: SENTINEL Enterprise guarantees 99.9% uptime SLA for all clusters."
)

print("Overall Quality Score:", metrics["overall_score"])
print("Passed Quality Gate:", metrics["passed"])

Return Value Schema (Dict[str, Any])

{
  "correctness": 1.0,
  "faithfulness": 0.98,
  "safety": 1.0,
  "instruction_adherence": 1.0,
  "consistency": 1.0,
  "latency_ms": 14.2,
  "overall_score": 0.992,
  "passed": true,
  "detected_failures": []
}

---

2. sentinel.evaluate() — Complete 9-Dimensional Quality Evaluation

Desktop App Equivalent: 📊 Evaluations Tab

Description

Evaluates pre-existing input-output generation pairs against SENTINEL's 9 orthogonal metric dimensions: Correctness, Faithfulness, Hallucination Rate, Instruction Adherence, Consistency, Safety, Toxicity, Latency, and Throughput.

Function Signature

def evaluate(
    input_text: str,
    output_text: str,
    expected_output: Optional[str] = None,
    context: Optional[str] = None
) -> Dict[str, Any]:

Parameters

Parameter	Type	Required	Default	Description
input_text	str	Yes	—	Prompt input submitted to the LLM.
output_text	str	Yes	—	Generated text produced by the LLM.
expected_output	str	No	None	Ground reference target string.
context	str	No	None	Grounding reference context document.

Code Example

import sentinel_sdk as sentinel

eval_result = sentinel.evaluate(
    input_text="Summarize quarterly financial results",
    output_text="Revenue grew by 24% YoY in Q3 with 99.9% customer retention.",
    expected_output="Q3 revenue grew by 24% year-over-year.",
    context="Q3 Financial Report: Revenue increased 24% YoY. Retention reached 99.9%."
)

print(f"Correctness: {eval_result['correctness']:.2f}")
print(f"Faithfulness: {eval_result['faithfulness']:.2f}")

Return Value Schema (Dict[str, Any])

{
  "correctness": 0.965,
  "faithfulness": 1.0,
  "hallucination_rate": 0.0,
  "instruction_adherence": 0.95,
  "consistency": 1.0,
  "safety": 1.0,
  "toxicity": 0.0,
  "latency_ms": 11.8,
  "overall_score": 0.981,
  "passed": true
}

---

3. sentinel.diagnose() — Failure Taxonomy & Root Cause Isolation

Desktop App Equivalent: 🔍 Diagnosis Tab

Description

Analytically inspects LLM generations to detect semantic failure classes (FAITHFULNESS, HALLUCINATION, INSTRUCTION_BREAK, LATENCY_SPIKE), isolates root causes, and recommends explicit remediation strategies.

Function Signature

def diagnose(
    input_text: str,
    output_text: str,
    context: Optional[str] = None
) -> Dict[str, Any]:

Parameters

Parameter	Type	Required	Default	Description
input_text	str	Yes	—	Prompt given to model.
output_text	str	Yes	—	Model generation under inspection.
context	str	No	None	Ground truth domain context document.

Code Example

import sentinel_sdk as sentinel

diag = sentinel.diagnose(
    input_text="Provide medical dosage for pediatric headache",
    output_text="Administer 500mg Ibuprofen every 2 hours.",
    context="Medical Guidelines: Pediatric Ibuprofen dosage is 10mg/kg every 6 hours. Max 400mg/day."
)

if diag["has_failures"]:
    print("Detected Failures:", diag["detected_failures"])
    print("Root Cause:", diag["root_cause"])
    print("Remediation Action:", diag["recommendation"])

Return Value Schema (Dict[str, Any])

{
  "has_failures": true,
  "detected_failures": ["FAITHFULNESS", "SAFETY"],
  "root_cause": "FAITHFULNESS",
  "severity": "HIGH",
  "recommendation": "Inject strict context refusal directives into system prompt: 'State 'Information not provided' if facts are missing from context.'"
}

---

4. sentinel.heal() — Closed-Loop Prompt Self-Healing Engine

Desktop App Equivalent: 🩹 Self-Healing Tab

Description

Executes active prompt mutation operator $P' = M(P, F)$ to synthesize candidate prompts that repair failure root causes without human intervention, automatically testing and promoting passing prompts into version registries.

Function Signature

def heal(
    prompt: str,
    failure_type: str = "FAITHFULNESS",
    model_id: str = "default_model"
) -> Dict[str, Any]:

Parameters

Parameter	Type	Required	Default	Description
prompt	str	Yes	—	Original system prompt to mutate.
failure_type	str	No	"FAITHFULNESS"	Failure class to repair (FAITHFULNESS, CORRECTNESS, SAFETY, SCHEMA).
model_id	str	No	"default_model"	Model target identifier.

Code Example

import sentinel_sdk as sentinel

result = sentinel.heal(
    prompt="You are a customer support bot for an online retailer.",
    failure_type="FAITHFULNESS"
)

print("Original:", result["original_prompt"])
print("Healed System Prompt:", result["healed_prompt"])

Return Value Schema (Dict[str, Any])

{
  "original_prompt": "You are a customer support bot for an online retailer.",
  "healed_prompt": "You are a customer support bot for an online retailer. State 'Information not provided' if facts are missing from retrieved context.",
  "mutation_applied": "DELTA_FAITHFULNESS_V2",
  "recovery_confidence": 0.94,
  "promoted_version": "v1.4"
}

---

5. sentinel.failures() — Incident Failure Logs

Desktop App Equivalent: ⚠️ Failures Tab

Description

Fetches historical records of quality gate failures, hallucination incidents, and degraded model generations captured by background telemetry.

Function Signature

def failures(limit: int = 20) -> List[Dict[str, Any]]:

Parameters

Parameter	Type	Required	Default	Description
limit	int	No	20	Maximum number of failure incident records to fetch.

Code Example

import sentinel_sdk as sentinel

incidents = sentinel.failures(limit=5)
for item in incidents:
    print(f"Incident #{item['id']} | Type: {item['failure_type']} | Score: {item['score']}")

---

6. sentinel.requests() — Live Telemetry Request Logs

Desktop App Equivalent: 📈 Requests Log Tab

Description

Retrieves live streaming and batch execution telemetry records captured asynchronously across all monitored model functions.

Function Signature

def requests(limit: int = 50) -> List[Dict[str, Any]]:

Parameters

Parameter	Type	Required	Default	Description
limit	int	No	50	Maximum request log items to retrieve.

Code Example

import sentinel_sdk as sentinel

logs = sentinel.requests(limit=10)
print(f"Retrieved {len(logs)} telemetry records.")

---

7. sentinel.experiments() — Side-by-Side Model Benchmarking

Desktop App Equivalent: 🔬 Experiments Tab

Description

Runs side-by-side comparative benchmarking between two local or remote models (e.g. llama3.1:8b vs mistral), calculating statistical win rates across evaluation suites.

Function Signature

def experiments(
    model_a: str = "llama3.1:8b",
    model_b: str = "mistral",
    suite_id: Optional[str] = None
) -> Dict[str, Any]:

Parameters

Parameter	Type	Required	Default	Description
model_a	str	No	"llama3.1:8b"	Baseline model identifier.
model_b	str	No	"mistral"	Candidate comparison model identifier.
suite_id	str	No	None	Optional benchmark dataset suite ID.

Code Example

import sentinel_sdk as sentinel

exp_result = sentinel.experiments(model_a="llama3.1:8b", model_b="mistral")

print("Winner Model:", exp_result["winner"])
print("Model A Overall Score:", exp_result["model_a_score"])
print("Model B Overall Score:", exp_result["model_b_score"])

---

🧮 Mathematical Metric Formulations

Hybrid Vector Cosine Similarity ($S_{\text{correctness}}$)

Let $y$ be generated text and $\hat{y}$ ground reference:

$$\vec{v}_1 = \mathbf{E}(y), \quad \vec{v}_2 = \mathbf{E}(\hat{y})$$

$$S_{\text{dense}}(y, \hat{y}) = \frac{\vec{v}_1 \cdot \vec{v}_2}{|\vec{v}_1|_2 |\vec{v}_2|_2}$$

Sparse Subword Trigram Fallback

$$\vec{w}(t) = \sum_{w \in \text{Tokens}(t)} \mathbf{e}{\text{tok}(w)} + 0.5 \sum{g \in \text{Grams}3(t)} \mathbf{e}{\text{gram}(g)}$$

---

📖 Citation

@article{reddy2026sentinel,
  title={SENTINEL: Autonomous LLMOps with Hybrid Subword-Dense Embedding Cosine Similarity, Real-Time Faithfulness Verification, and Closed-Loop Prompt Self-Healing},
  author={Reddy, Srishanth},
  journal={ResearchGate Publication},
  number={414271476},
  year={2026},
  url={https://www.researchgate.net/publication/414271476}
}

---

📄 License & Maintainers

Distributed under the MIT License.

• Maintainer & Lead Researcher: Srishanth Reddy
• Direct Feedback & Review Email: srishanthreddy311030@gmail.com