pysad 0.8.0

English · 简体中文

One pass. Every outlier.

Streaming anomaly detection in Python. PySAD scores each data point when it arrives,
with streaming detectors, calibrated alerts and streaming evaluation.

Documentation · Getting started · How-to guides · Benchmarks · Detectors · API · Live demo · Try it in Colab

_{A Robust Random Cut Forest scores three synthetic sensors one point at a time. Conformal calibration converts each score into the fraction of recent scores that it exceeds, and raises an alert at 0.995.}

Why PySAD

A batch detector needs the whole dataset before it can score a point. A stream does not give you the whole dataset. Points arrive one at a time, the data drifts (its distribution changes over time), and you must decide about each point before the next point arrives. PySAD is built for this setting. More than 60 publications on Google Scholar cite it, and 30+ public repositories on GitHub use it.

Batch outlier detectors such as PyOD train on a fixed dataset and must train again to use new data. PySAD detectors learn from every point, and PySAD can still run PyOD models on a stream with ReferenceWindowModel. General online learning frameworks such as River treat anomaly detection as one module among many. PySAD is only about streaming anomaly detection: its detectors come with calibration, postprocessing, ensembling and evaluation.

Installation

pip install pysad

Or with Conda / Mamba / Pixi:

conda install -c conda-forge pysad

PySAD supports Python 3.10 to 3.14 on Linux, macOS and Windows. Every detector works after installation, with no optional extras.

To install the latest development version from master:

pip install git+https://github.com/selimfirat/pysad.git

For an editable installation with development dependencies, follow the contributing guide.

Quick start

Score a stream point by point, and measure the detector while it runs:

import numpy as np
from pysad.evaluation import AUROCMetric
from pysad.models import LODA
from pysad.utils import ArrayStreamer

# A stream of 2,000 points with 4 features. Every 50th point is an anomaly.
rng = np.random.default_rng(42)
X = rng.normal(size=(2000, 4))
y = np.zeros(2000, dtype=int)
y[::50] = 1
X[y == 1, :2] += 3  # the code shifts the anomalies in two of the four features

model = LODA(random_state=0)
metric = AUROCMetric()

for x, label in ArrayStreamer().iter(X, y):
    score = model.fit_score_partial(x)  # score x, then learn from it
    metric.update(label, score)

print(f"AUROC: {metric.get():.3f}")  # prints AUROC: 0.960

Turn scores into alerts

A higher score means a more anomalous point, but raw scores have no fixed scale. The conformal calibrator converts each score into the fraction of recent scores that it exceeds. Continuing the example above, raise an alert when this value is close to 1:

from pysad.transform.probability_calibration import ConformalProbabilityCalibrator

model = LODA(random_state=0)
calibrator = ConformalProbabilityCalibrator(window_size=500)

alerts = []
for t, x in enumerate(ArrayStreamer().iter(X)):
    calibrated = calibrator.fit_transform_partial(model.fit_score_partial(x))
    if t >= 200 and calibrated >= 0.99:  # skip a short warm-up
        alerts.append(t)

# Prints: 20 alerts, 15 of them true anomalies
print(f"{len(alerts)} alerts, {y[alerts].sum()} of them true anomalies")

Stream your own data

A detector takes one NumPy array per point, so any loop works: a CSV reader, a socket or a message queue consumer.

import json

for message in consumer:  # for example, a Kafka consumer
    x = np.asarray(json.loads(message.value)["features"], dtype=float)
    score = model.fit_score_partial(x)

ArrayStreamer, PandasStreamer and PolarsStreamer replay arrays and pandas or Polars DataFrames as streams for experiments.

Use a PyOD model on a stream

Install PyOD first with pip install pyod.

from pyod.models.iforest import IForest
from pysad.models.integrations import ReferenceWindowModel

# Train a batch Isolation Forest on the last 200 points, again every 50 points.
model = ReferenceWindowModel(IForest, window_size=200, sliding_size=50, initial_window_X=X[:100])
for x in ArrayStreamer().iter(X[100:]):
    score = model.fit_score_partial(x)

The getting started tutorial explains each step. The how-to guides cover alerts, evaluation, pipelines, seasonal series, production use and custom detectors.

Detectors

Which detector to start with

These starting points come from the design of each method. They are not benchmark rankings. The benchmark gives the results of each multivariate detector on 17 datasets. Compare two or three detectors on your own data, as the guide to choosing a detector shows.

All detectors

Multivariate

Univariate

ReferenceWindowModel and OneFitModel run any PyOD detector on a stream. The detector catalog lists the input, setup and memory of each detector, and the API reference lists every parameter.

Benchmarks

PySAD benchmarks its detectors on labeled data. The 10 multivariate detectors run on 17 datasets from the ODDS repository and on 8 real streams with drift. The univariate detectors run on the series of the Numenta Anomaly Benchmark (NAB) and of the UCR archive. Each detector runs once on each dataset, with its default parameters, and scores each point before it learns from it (fit_score_partial).

The benchmark results page shows all the results. The evaluation methodology explains the steps, the metrics and the limits of the results, and the datasets page describes the data. To run the benchmark, use python benchmarks/benchmark.py, as Run the benchmark shows. On each pull request that changes a detector, CI runs a short form of the benchmark and lists the results that moved.

Beyond detectors

Contributing

We welcome issues and pull requests, and we aim to reply within a few days. Ask questions and share your projects in Discussions. For a first contribution, choose a good first issue. The contributing guide lists the steps from the first clone to an open pull request.

PySAD follows semantic versioning and trunk-based development. Each change reaches master through a short-lived branch and a pull request that CI checks, and CI runs every example.

Thanks to everyone who has reported a bug, fixed one, or improved the docs.

Citing PySAD

If you use PySAD in a scientific publication, cite the paper:

@article{pysad,
  title={PySAD: A Streaming Anomaly Detection Framework in Python},
  author={Yilmaz, Selim F and Kozat, Suleyman S},
  journal={arXiv preprint arXiv:2009.02572},
  year={2020}
}

To cite a specific version of the software, use its Zenodo DOI. The DOI 10.5281/zenodo.22983312 always points to the latest release. That page also lists a separate DOI for each release.

License

PySAD is free software under the BSD 3-Clause License.

If PySAD is useful to you, a star helps other people find it.
To hear about new detectors and releases, choose Watch → Custom → Releases.