tadmetric 0.1.7

A lightweight evaluation toolkit for time-series anomaly detection

tadmetric

Lightweight evaluation metrics for time-series anomaly detection.

tadmetric gives you a small, practical API for evaluating anomaly detection results from either binary predictions or raw anomaly scores. The main workflow is centered on evaluator(...), which lets you reuse one score array across multiple metrics and threshold-search runs.

Why tadmetric

• Small API surface for common TSAD evaluation workflows
• Supports both y_pred and y_score based evaluation
• Fast best-threshold search for repeated experiments
• Includes point-wise, event-wise, point-adjusted, composite, delay, and curve metrics
• Extensible metric registry for custom research metrics

Installation

pip install tadmetric

Python 3.9+ is supported.

Quick Start

1. Evaluate binary predictions

import tadmetric as tm

y_true = [0, 0, 1, 1, 1, 0, 0]
y_pred = [0, 0, 1, 1, 0, 0, 0]

result = tm.evaluate(y_true, y_pred, metrics=("point", "point_adjusted", "composite"))

print(result["point"].asdict())
print(result["point_adjusted"].f1)
print(result["composite"].f1)

2. Evaluate scores with a reusable evaluator

import tadmetric as tm

y_true = [0, 0, 1, 1, 1, 0, 0]
y_score = [0.1, 0.2, 0.4, 0.9, 0.7, 0.1, 0.0]

e = tm.evaluator(y_true, y_score)

point = e.point_wise(thr=0.5)
adjusted = e.point_adjusted(thr=0.3)
adjusted_k = e.point_adjusted_k(thr=0.3, k=30)
composite = e.composite(thr=0.5)

print(point.asdict())
print(adjusted.f1, adjusted_k.f1, composite.f1)

3. Search for the best threshold

import tadmetric as tm

y_true = [0, 0, 1, 1, 1, 0, 0]
y_score = [0.1, 0.2, 0.4, 0.9, 0.7, 0.1, 0.0]

e = tm.evaluator(y_true, y_score)
best = e.best(metric="composite", progress=True)

print(best.threshold)
print(best.f1)
print(best.evaluation["composite"].asdict())

progress=True writes to stderr. If you want to see it while running tests, use pytest -s.

Main API

For most workflows, these are the core entry points:

• evaluator(y_true, y_score)
• evaluate(y_true, y_pred, ...)
• evaluate_scores(y_true, y_score, threshold=...)
• e.point_wise(thr=...)
• e.point_adjusted(thr=...)
• e.point_adjusted_k(thr=..., k=...)
• e.composite(thr=...)
• e.best(metric=...)
• available_metrics()
• describe_metrics()
• api_overview()
• register_metric(...)

Included Metrics

Core precision / recall / F1

• Point-wise: point_precision, point_recall, point_f1
• Point-adjusted: point_adjusted_precision, point_adjusted_recall, point_adjusted_f1
• Point-adjusted %K: point_adjusted_k_precision, point_adjusted_k_recall, point_adjusted_k_f1
• Composite: composite_precision, composite_recall, composite_f1
• Event-wise: event_precision, event_recall, event_f1

Delay-aware metrics

• time_to_detect
• mean_time_to_detect
• median_time_to_detect
• missed_detection_rate

Thresholding and utilities

• threshold_by_quantile
• threshold_by_topk
• threshold_by_best_f1
• search_best_f1_threshold
• apply_hysteresis
• scores_to_binary
• binary_to_intervals
• intervals_to_binary
• merge_intervals

Curve-based metrics

• precision_recall_curve
• roc_curve
• auc_pr
• auc_roc

Metric Semantics

• Point-wise metrics treat each timestamp independently.
• Event-wise metrics treat each contiguous anomaly region as one event.
• Point-adjusted metrics credit a whole event once it is detected.
• Point-adjusted %K metrics credit an event when at least K% of the event is detected.
• Composite metrics use point-wise precision and event-based recall.
• Interval semantics are half-open: [start, end).
• Event matching defaults to overlap > 0.
• zero_division defaults to 0.0.

Discoverability

If you are opening the package for the first time, these helpers are useful:

import tadmetric as tm

print(tm.api_overview())
print(tm.available_metrics())
print([spec.asdict() for spec in tm.describe_metrics()])

Development

pip install -e .[dev]
pytest -q

Release Checklist

pytest -q
python -m build --no-isolation
python -m twine check dist/*