scryml 0.2.0

Predict infrastructure failure states from a stream of metrics

scry ml

Scry predicts infrastructure failure states from a stream of metrics. It learns what healthy looks like, scores each resource's reconstruction error against a threshold calibrated on healthy data, and alarms before degradation; an incident-validation harness measures how early (the detection lead time). It also forecasts where the metrics are headed and discovers operational states by unsupervised clustering. It is data-source agnostic and runs offline. No trained weights ship.

Currently being used for LogicMonitor metric predictions. The default path reads Parquet or CSV from local files or object storage. Everything normalizes to one canonical long-format table, so any metric source works once it is in that shape.

Shout out to my MTG heads.

The X-DEC model	A dual-encoder temporal VAE plus deep embedded clustering, pure PyTorch, no cloud dependencies. Trains on your own windowed metrics from scratch.
Reconstruction anomaly signal	The validated serving signal: per-window reconstruction error as a ratio of a threshold baked from healthy windows (scripts/bake_serving_threshold.py), served at /anomaly/reconstruction. Windows too short or without numerical coverage are reported not-scored, never padded into a false alarm.
Operational-state discovery	Deep embedded clustering discovers five nominal states: NORMAL, PRE_SCALE, PRE_FAILURE, ACTIVE_DEGRADATION, ANOMALY. The labels are unsupervised and not yet grounded on labeled incidents, so on healthy-trained models cluster-derived alerting stays disabled and the reconstruction signal is authoritative.
Forecasting	An optional Chronos layer (scryml[forecast]) projects where each metric is headed across multiple horizons, kept behind an extra so the core stays offline-capable.
Data-source agnostic	Read Parquet or CSV from local disk or object storage (S3, GCS, ADLS, MinIO) through DuckDB. A LogicMonitor REST exporter lives behind scryml[logicmonitor].
Coverage and quality checks	Before training or serving, report which profile features are present or missing and whether the data is fresh and gap-free, computed directly over the object store with scripts/validate_data.py.
Incident-validation harness	Score a trained model against a labeled incident capture: per-window reconstruction error thresholded on healthy data only, reporting detection lead time (how early the alarm fires before onset) with scripts/validate_incident.py.
A small HTTP service	FastAPI endpoints for anomaly scoring, prediction, and forecasting: /anomaly/reconstruction, /anomaly/reconstruction/lookup, /predict, /predict/lookup, /forecast. /drift, /anomaly, and /accuracy need operator-wired detectors and serve 503 until configured.
Schema-checked serving	Each trained model carries its feature schema (names, order, profile, normalization). The API aligns incoming metrics by name, so a coverage gap between sources is a checked contract, not a silent misalignment.
Bring your own data	One canonical schema: resource, metric, timestamp, value, plus optional host and datasource fields. Drop your metrics into that table and train. Profiles may be purely numerical or mix numerical and categorical features. No real telemetry or weights are included.

---

Install

pip install scryml                  # core: the model and the API
pip install "scryml[forecast]"      # add Chronos forecasting
pip install "scryml[logicmonitor]"  # add the LogicMonitor REST exporter

Or from source with every extra:

git clone https://github.com/ryanmat/scry && cd scry
uv sync --all-extras

Quickstart

End to end on the bundled synthetic sample, no cloud:

# extract windowed features (the sample is dated 2026-01-01)
python scripts/extract_features.py --data examples/sample_data/metrics.parquet \
  --start 2026-01-01 --end 2026-01-02 --profile kubernetes \
  --output data/training_data.npz

# train a model, then serve it
python scripts/train_model.py --data data/training_data.npz --output models/xdec_model.pt
MODEL_PATH=models/xdec_model.pt uvicorn scry.api.main:app --port 8000

curl localhost:8000/health
curl localhost:8000/clusters

Full walkthrough: examples/quickstart.md.

Documentation

• Architecture: the model, the data seam, and how the pieces fit together.
• Data contract: the canonical metric schema.
• Ingestion: the object store and the routes that land data in it.
• Roadmap: where Scry is headed.
• Training: training locally or on your own orchestrator.

Built with uv, ruff, and pytest on Python 3.10 and up.

License

Apache-2.0.

WAKE UP TO FIND OUT THAT YOU ARE THE EYES OF THE WORLD