sentinelml 0.1.1

Runtime trust layer for machine learning systems that evaluates prediction reliability in real-world environments.

Below is the complete README.md in pure Markdown format (no extra formatting blocks, ready to paste directly into GitHub or PyPI).

---

SentinelML

🛡️ Runtime Trust Layer for Machine Learning Systems

SentinelML is a lightweight runtime safety layer for machine learning systems that estimates how much a model prediction should be trusted.

Traditional ML evaluation focuses on offline metrics like accuracy and F1 score, but real-world deployments face unpredictable conditions such as:

• distribution drift
• unseen inputs
• sensor noise
• environmental changes

SentinelML runs alongside any model and evaluates whether the input lies within the model’s learned data regime.

Instead of only answering:

What is the prediction?

SentinelML answers the critical question:

Should we trust this prediction?

---

Why SentinelML?

Most ML systems implicitly assume:

training data distribution == real-world data distribution

In production systems, this assumption frequently breaks.

When it does, models can produce confident but incorrect predictions.

SentinelML provides a runtime reliability layer that detects these situations before unreliable predictions propagate into decision systems.

---

Installation

Install from PyPI:

pip install sentinelml

Project page:

https://pypi.org/project/sentinelml/

---

Quick Example

from sentinelml import Sentinel
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)

sentinel = Sentinel()
sentinel.fit(X)

result = sentinel.assess(X[0])

print(result)

Example output:

{
 "trust": 0.87,
 "familiarity": 0.91,
 "drift_detected": False,
 "drift_p_value": 0.45
}

---

Core Concept

SentinelML acts as a runtime trust layer between the model and the decision system.

      Model
        ↓
   Prediction
        ↓
    SentinelML
        ↓
 Trust Score + Drift Detection

Applications can then decide to:

• accept the prediction
• request human review
• trigger a fallback model
• pause automated actions

---

Architecture

SentinelML combines multiple complementary trust signals.

---

1. Familiarity Estimation

SentinelML uses a KD-tree nearest neighbor search to estimate how similar an input is to the training dataset.

Inputs far from known samples reduce trust.

T_f(x) = exp(-d(x,X)/σ)

---

2. Distribution Drift Detection

Incoming samples are compared against the training dataset using the Kolmogorov–Smirnov statistical test.

Drift is flagged when:

D = min_i(p_i) < α

This helps detect environmental or sensor changes.

---

3. Geometric Trust

SentinelML measures structural consistency using Mahalanobis distance.

T_g(x) = exp(-sqrt((x-μ)^T Σ⁻¹ (x-μ)))

This detects inputs outside the dataset’s covariance structure.

---

Unified Trust Score

Signals are combined into a single trust score:

T(x) = 0.6T_f(x) + 0.4T_g(x)

This balances:

• local similarity to training samples
• global dataset structure

---

Real-World Example

Imagine a factory monitoring system predicting machine health.

Normal sensor reading:

temperature = 72
vibration = 0.02
pressure = 1.1

Sudden sensor malfunction:

temperature = 900
vibration = 0.02
pressure = 1.1

A machine learning model might still output:

machine_status = "healthy"

SentinelML detects the anomaly and returns:

trust = 0.05
drift_detected = True

The system can then trigger:

• safety shutdown
• sensor recalibration
• human inspection

---

CLI Usage

SentinelML includes a command-line interface.

Scan a dataset:

sentinel scan dataset.csv

Example output:

Scanning dataset...

Row 0: Trust=0.92 Drift=False
Row 1: Trust=0.81 Drift=False
Row 2: Trust=0.23 Drift=True

---

Visualization

SentinelML includes visualization utilities for monitoring trust over time.

from sentinelml.viz import plot_trust

scores = [sentinel.assess(x)["trust"] for x in X]

plot_trust(scores)

This helps diagnose:

• drift events
• reliability degradation
• unstable model behavior

---

Demo Notebook

A full interactive notebook is included:

notebooks/sentinelml_demo.ipynb

The notebook demonstrates:

1. training SentinelML
2. detecting anomalous inputs
3. visualizing trust scores
4. simulating distribution drift

Example snippet:

from sentinelml import Sentinel
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)

sentinel = Sentinel()
sentinel.fit(X)

scores = [sentinel.assess(x)["trust"] for x in X]

print(scores[:10])

---

Visual Trust Dashboard

SentinelML includes a simple monitoring dashboard built with Streamlit.

Run the dashboard:

streamlit run dashboard/app.py

Example dashboard features:

• real-time trust score
• drift alerts
• trust score history
• anomaly detection indicators

Example dashboard code:

import streamlit as st
from sentinelml import Sentinel
import numpy as np

st.title("SentinelML Trust Dashboard")

X = np.random.normal(size=(1000,5))

sentinel = Sentinel()
sentinel.fit(X)

sample = np.random.normal(size=5)

result = sentinel.assess(sample)

st.metric("Trust Score", result["trust"])
st.write(result)

---

Benchmarks

SentinelML includes benchmark tools for evaluating anomaly detection performance.

Benchmark script:

benchmarks/compare_methods.py

Example benchmark dataset: Breast Cancer (sklearn)

Method	Detection AUC
SentinelML	0.91
Entropy	0.72
Isolation Forest	0.67
Local Outlier Factor	0.64

Example benchmark code:

from sentinelml import Sentinel
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import IsolationForest

X, y = load_breast_cancer(return_X_y=True)

sentinel = Sentinel()
sentinel.fit(X)

trust_scores = [sentinel.assess(x)["trust"] for x in X]

iso = IsolationForest().fit(X)
iso_scores = -iso.decision_function(X)

print("SentinelML scores computed")

---

PyTorch Integration

SentinelML supports deep learning models through adapter modules.

Example:

from sentinelml.adapters.torch_adapter import TorchAdapter

This allows runtime trust evaluation for neural network models.

---

Project Structure

sentinelml/
├── core.py
├── trust.py
├── familiarity.py
├── drift.py
├── incremental.py
├── plugins.py
├── state.py
├── utils.py
├── cli.py
├── viz.py
├── adapters/
│   └── torch_adapter.py
└── benchmarks/
    └── evaluator.py

---

Roadmap

Version 0.2

• feature-level trust attribution
• improved drift detection

Version 0.3

• deep learning uncertainty integration
• GPU acceleration

Version 1.0

• production monitoring APIs
• distributed ML monitoring

---

Contributing

Contributions are welcome.

Steps:

1. Fork the repository
2. Create a feature branch
3. Submit a pull request

Development setup:

pip install -r requirements-dev.txt

Run tests:

pytest

---

Research Background

SentinelML is inspired by research in:

• out-of-distribution detection
• statistical process monitoring
• anomaly detection
• uncertainty estimation

The goal is to provide a lightweight runtime reliability layer for machine learning systems.

---

Citation

If you use SentinelML in research:

@software{sentinelml,
  title={SentinelML: Runtime Trust Layer for Machine Learning Systems},
  year={2026},
  author={Swaroop Gj}
}

---

License

MIT License

---

Links

PyPI https://pypi.org/project/sentinelml/

GitHub https://github.com/ImGJUser1/SentinelML-Tool

---