stayready 0.3.0

Compute and push AI model drift metrics to Compass StayReady. Zero-dependency core; optional local metric computation and an Evidently adapter.

StayReady Python SDK

Push drift events and audit entries from your MLOps pipeline to Compass StayReady — continuous AI governance monitoring for regulated sectors.

Install

pip install stayready                # zero-dependency client
pip install "stayready[metrics]"     # + local metric computation (adds numpy)

The core client depends on nothing outside the standard library, on purpose: it drops into a locked-down environment without dragging a dependency tree through a security review. Metric computation is an opt-in extra.

Three ways to use it

You have	Use	Needs
Metrics already (any tool)	sr.drift(...)	nothing
Raw data, no drift tooling	Monitor	[metrics] extra
Evidently already running	from_evidently(...)	nothing

Quick start

import os
from stayready import StayReady

sr = StayReady(
    api_key=os.environ["STAYREADY_API_KEY"],
    model_id="your-model-uuid",  # from your StayReady dashboard
)

# Report drift when your monitoring detects it — StayReady computes severity
# itself from value/threshold/direction and this metric's own history; you
# don't set it.
sr.drift(
    type="data_drift",              # data_drift | concept_drift | performance_degradation | regulatory_change
    metric="PSI",
    value=0.28,
    threshold=0.20,
    description="Population Stability Index exceeded threshold on income feature.",
    affected_domains=["Data Quality", "Model Monitoring"],
    action_required="Retrain on recent data and re-validate before next production cycle.",
)

# Log lifecycle events to the immutable audit trail
sr.audit("model_retrained", "Retrained on Q2 data, Gini improved to 0.61", actor="ml-pipeline")

Severity is computed server-side by comparing value against threshold and this model+metric's own trailing baseline — not something you set. Critical and High results trigger an email alert (and webhook, if configured), deduped so repeated near-identical events don't re-notify, and can flag your linked GovernReady audit for re-assessment.

Most metrics are "higher is worse" (PSI, KS, error rate) — the default. If a drop is what's bad for your metric (Gini, accuracy, F1), pass direction="lower_is_worse" on the first call you ever make for that model+metric; it registers the metric's polarity and is ignored on later calls, so you can't accidentally flip it mid-stream.

Integration examples

Evidently AI

psi = report.as_dict()["metrics"][0]["result"]["dataset_drift_share"]
sr.drift(type="data_drift", metric="PSI", value=psi,
         threshold=0.2, description="Evidently reported this drift share.")

Airflow (post-training validation)

def report_validation(**ctx):
    gini = ctx["ti"].xcom_pull(key="gini")
    sr.drift(type="performance_degradation", metric="Gini", value=gini,
             threshold=0.55, direction="lower_is_worse",
             description="Gini after retrain.")

Requirements

Python 3.9+. Zero dependencies (standard library only).

Support

[e-mail] · stayready.dendrons.ai

Compute metrics from your own data (Monitor)

If you don't already have a drift tool, Monitor computes the standard metrics locally and pushes them. Only aggregate values leave your infrastructure — a handful of floats per column. No raw records, no feature values, no model outputs. That property is usually the deciding factor in a regulated buyer's security review.

from stayready import StayReady, Monitor

sr = StayReady(api_key=..., model_id=...)
mon = Monitor(sr)

# Always dry-run first: computes and returns, sends nothing.
print(mon.check_drift(reference=train_df, current=recent_df, dry_run=True))

# Input drift, one metric per column (psi | ks | js | chisquare)
mon.check_drift(reference=train_df, current=recent_df, metric="psi")

# Model quality. Gini is pushed as lower_is_worse automatically —
# a *drop* is the problem, and getting that backwards inverts severity.
mon.check_performance(y_true=y, y_pred=preds, y_score=scores)

reference and current accept a pandas DataFrame or a plain {"column": [values]} dict — pandas is not required.

Metrics are verified against SciPy: KS matches scipy.stats.ks_2samp, AUC matches scipy.stats.mannwhitneyu, and Jensen-Shannon matches scipy.spatial.distance.jensenshannon (base 2), each to 1e-9.

This is a reference implementation, not a drift-detection product. It computes standard metrics competently; it does not schedule runs, store baselines, or manage windows. If you already run Evidently, use the adapter below instead of computing the same numbers twice.

Already using Evidently? (from_evidently)

from evidently import Report
from evidently.presets import DataDriftPreset
from stayready import StayReady, from_evidently

snapshot = Report([DataDriftPreset()]).run(current_data=cur, reference_data=ref)
from_evidently(snapshot, client=StayReady(api_key=..., model_id=...))

Accepts a Report, a snapshot, its .dict(), or a JSON string. Requires no extra install — it only reads Evidently's output structure.

It handles a trap you'd otherwise hit. Evidently's stattests don't share a direction convention and the JSON doesn't say which applies. Distances (psi, jensenshannon, wasserstein, hellinger, kl_div, ed) report higher = more drift. Test-based stattests (ks, chisquare, z, g_test, TVD, t_test, mannw, anderson, cramer_von_mises, es) report a p-value, where lower = more drift. Forwarding a p-value as higher-is-worse inverts severity exactly: real drift scores healthy, and a healthy model pages someone at 3am. TVD returns a p-value despite the name.

The direction map was built by running each stattest against known-drifted and known-clean samples and recording which way the value actually moved. For a stattest it doesn't recognise, the adapter skips with a warning rather than guessing — override with unknown_method="higher_is_worse" (or "lower_is_worse") when you know which applies.