The Third Eye

AI model health and governance control plane for Databricks

Watch. Understand. Act.

Databricks provides the signals. The Third Eye turns them into decisions.

What is The Third Eye?

The Third Eye is a Databricks-native AI health and governance control plane.

It does not replace Model Serving, MLflow, Unity Catalog, Lakehouse Monitoring, Unity Gateway, or Workflows. It sits above those primitives and turns their signals into an explainable operating view:

Databricks signals
      │
      ├── monitoring
      ├── inference / trace
      ├── lineage
      ├── usage / cost
      └── governance / guardrails
      │
      ▼
The Third Eye
      │
      ├── normalize
      ├── correlate
      ├── score
      ├── explain
      └── recommend
      │
      ▼
governed model health
      │
      ├── dashboard
      ├── Genie
      └── Teams / Slack

Positioning: not another drift detector. Not another dashboard. The Third Eye is the decision layer above the native Databricks AI stack.

Why this exists

AI fleets fail in ways that are difficult to diagnose from a single metric.

A model can look healthy in one system while another already shows a warning:

• feature drift is rising
• quality is deteriorating
• an upstream table changed
• token or endpoint cost spiked
• a gateway policy was breached
• model criticality makes the event more urgent

The Third Eye correlates these signals into a single evidence-backed health view.

The core distinction

Traditional monitoring:
metric → threshold → alert

The Third Eye:
signals → correlation → context → risk → explanation → action

Core capabilities

Architecture

┌──────────────────────── Databricks native layer ────────────────────────┐
│ Unity Catalog · MLflow · Monitoring · Inference / Trace · Lineage       │
│ Unity Gateway · Usage · Guardrails · Databricks Workflows               │
└────────────────────────────────┬────────────────────────────────────────┘
                                 │
                                 ▼
                    ┌──────────────────────────┐
                    │ Discovery & Sync         │
                    │ model / asset registry  │
                    └─────────────┬────────────┘
                                  ▼
                    ┌──────────────────────────┐
                    │ Signal Normalization     │
                    │ common signal index      │
                    └─────────────┬────────────┘
                                  ▼
                    ┌──────────────────────────┐
                    │ Correlation & Scoring    │
                    │ health + confidence      │
                    └─────────────┬────────────┘
                                  ▼
                    governance.model_health.*
                         │        │        │
                         ▼        ▼        ▼
                       AI/BI    Genie   Alerts
                                  │
                                  ▼
                        Root cause / remediation

Repository structure

The repository remains the full Databricks implementation and deployment reference:

adapters/         native-signal adapters
discovery/        model and lineage discovery
scoring/          correlation and health scoring jobs
remediation/     root-cause and remediation generation
passport/         model passport generation
sql/              Unity Catalog bootstrap + governed tables
genie/            Genie instructions
copilot_studio/  notification / topic notes
workflows/       Databricks Workflow definitions
demo/             product demonstration assets
v2_stretch/       planned governed-autonomy components
third_eye/       reusable Python scoring primitives

Quickstart

Install the reusable Python package

pip install the-third-eye

Use the portable scoring kernel

from third_eye.scoring import health_score, health_tier

score = health_score(
    drift=0.30,
    quality=0.92,
    cost=0.15,
    guardrail=0.05,
)

print(f"Health: {score:.2f}")
print(f"Tier:   {health_tier(score)}")

The package is intentionally small. It exposes deterministic scoring primitives while the Databricks-specific orchestration remains in the repository.

Databricks integrations

pip install "the-third-eye[databricks]"

This optional extra installs the Databricks SDK, MLflow, and Spark dependencies used by the workspace integration layer.

Run locally

The Databricks jobs support the existing mock-mode development path:

set THIRD_EYE_MOCK_MODE=true
python scoring/compute_health_score.py
python scoring/correlate_signals.py

PowerShell:

$env:THIRD_EYE_MOCK_MODE = "true"
python scoring/compute_health_score.py
python scoring/correlate_signals.py

The health score

The central composite calculation is:

health_score = 100 - (
    w1 · drift
  + w2 · (1 - quality)
  + w3 · cost
  + w4 · guardrail
) · criticality_weight

The portable Python implementation keeps this kernel deterministic and independent of Databricks runtime concerns.

Evidence before action

The Third Eye follows a read-first, evidence-before-action principle.

A single weak signal should not automatically become a production action.

signal
  ↓
corroborating evidence
  ↓
criticality context
  ↓
risk / confidence
  ↓
incident
  ↓
root cause
  ↓
recommended action

High-impact production actions remain policy-controlled.

Security and governance

The control plane is designed around governed access rather than copying source systems.

Key principles:

• native Databricks tables remain the source of record
• normalized signals store pointers and snapshots rather than duplicating raw telemetry
• secrets should be resolved through Databricks Secrets
• high-impact automation remains policy-controlled
• evidence and confidence accompany health decisions

Roadmap

V1

Discovery → normalization → correlation → scoring → diagnosis → governed outputs.

V2

Governed autonomy:

health degradation
      ↓
candidate retraining
      ↓
champion / challenger
      ↓
approval gate
      ↓
promotion / rollback

Planned work includes blast-radius analysis, multi-workspace federation, silent-failure detection, richer compliance reporting, and cost-of-drift business impact.

Development

git clone https://github.com/dcsgod/theThirdEye.git
cd theThirdEye

python -m pip install -e ".[dev]"
ruff check .
pytest
python -m build
python -m twine check dist/*

Publishing to PyPI

The repository includes a GitHub Actions workflow for PyPI Trusted Publishing.

GitHub Release
      ↓
build sdist + wheel
      ↓
twine check
      ↓
PyPI Trusted Publishing

For the first release, configure a PyPI Trusted Publisher with:

Owner:       dcsgod
Repository:  theThirdEye
Workflow:    publish.yml
Environment: pypi

Then create a GitHub release such as:

v0.1.0

The workflow publishes the built distributions to PyPI without a long-lived API token.

License

Apache-2.0.