driftvane

Compose drift detectors for RAG and agent systems.

Most drift libraries are either tabular-only (Evidently, DataDrift) or are platforms that want you to ship telemetry to their backend (Phoenix, Arize). driftvane is a small Python library that lets you wire up multiple drift signals — embedding, retrieval, response, latency — into one report. No server, no UI, no telemetry. Plug it into a Lambda or Glue job, get a pandas.DataFrame or a JSON dict back.

Install

pip install driftvane
# optional
pip install "driftvane[pandas]"            # to_pandas()
pip install "driftvane[external-response]" # delegate response scoring to context-drift-detector-py

Quickstart

import numpy as np
from driftvane import (
    DriftReport,
    EmbeddingDrift,
    RetrievalDrift,
    ResponseDrift,
    LatencyDrift,
)
from driftvane.detectors.response import Triple

ref_emb = np.load("reference_query_embeddings.npy")  # (n, 768)
cur_emb = np.load("current_query_embeddings.npy")

report = DriftReport.from_signals([
    EmbeddingDrift(threshold=0.1).compute(ref_emb, cur_emb),
    RetrievalDrift(k=10, threshold=0.3).compute(ref_top_k, cur_top_k),
    ResponseDrift(threshold=0.15).compute(ref_triples, cur_triples),
    LatencyDrift(p_threshold=0.01).compute(ref_latencies, cur_latencies),
])

if report.any_drifted():
    print(report.to_pandas())

Or fail a CI job when retrieval moves too much:

from driftvane import DriftAlert

try:
    report.alert_if({"retrieval_jaccard_at_10": 0.2})
except DriftAlert as e:
    sys.exit(f"drift gate failed: {e}")

Detectors

Each detector returns a DriftSignal(name, value, threshold, drifted, metadata). DriftReport collects them.

What it does NOT do

• No server. No UI. No telemetry shipping.
• No tabular feature drift — use DataDrift for KS/PSI on classical features.
• No live trace ingestion or OTel collection — point this at parquet/numpy arrays you already have.
• No causal root-cause analysis. It tells you that drift is there, not why.
• No model retraining triggers — emit your own when report.any_drifted().

Why not Phoenix / Arize / Evidently / Ragas?

Status

v0.1 — alpha. The four detectors above work and have tests. Public API may change before v1.0. Issues and PRs welcome.