Skip to main content

Orion is a machine learning library built for data generated by satellites.

Project description

“DAI-Lab” An open source project from Data to AI Lab at MIT.

“Orion”

Development Status PyPi Shield Tests Downloads Binder

Orion

Overview

Orion is a machine learning library built for unsupervised time series anomaly detection. With a given time series data, we provide a number of “verified” ML pipelines (a.k.a Orion pipelines) that identify rare patterns and flag them for expert review.

The library makes use of a number of automated machine learning tools developed under Data to AI Lab at MIT.

Recent news: Read about using an Orion pipeline on NYC taxi dataset in a blog series:

Part 1: Learn about unsupervised time series anomaly detection Part 2: Learn how we use GANs to solving the problem? Part 3: How does one evaluate anomaly detection pipelines?

Notebooks: Discover Orion through colab by launching our notebooks!

Quickstart

Install with pip

The easiest and recommended way to install Orion is using pip:

pip install orion-ml

This will pull and install the latest stable release from PyPi.

In the following example we show how to use one of the Orion Pipelines.

Fit an Orion pipeline

We will load a demo data for this example:

from orion.data import load_signal

train_data = load_signal('S-1-train')
train_data.head()

which should show a signal with timestamp and value.

    timestamp     value
0  1222819200 -0.366359
1  1222840800 -0.394108
2  1222862400  0.403625
3  1222884000 -0.362759
4  1222905600 -0.370746

In this example we use lstm_dynamic_threshold pipeline and set some hyperparameters (in this case training epochs as 5).

from orion import Orion

hyperparameters = {
    'keras.Sequential.LSTMTimeSeriesRegressor#1': {
        'epochs': 5,
        'verbose': True
    }
}

orion = Orion(
    pipeline='lstm_dynamic_threshold',
    hyperparameters=hyperparameters
)

orion.fit(train_data)

Detect anomalies using the fitted pipeline

Once it is fitted, we are ready to use it to detect anomalies in our incoming time series:

new_data = load_signal('S-1-new')
anomalies = orion.detect(new_data)

:warning: Depending on your system and the exact versions that you might have installed some WARNINGS may be printed. These can be safely ignored as they do not interfere with the proper behavior of the pipeline.

The output of the previous command will be a pandas.DataFrame containing a table of detected anomalies:

        start         end  severity
0  1394323200  1399701600  0.673494

Leaderboard

In every release, we run Orion benchmark. We maintain an up-to-date leaderboard with the current scoring of the verified pipelines according to the benchmarking procedure.

We run the benchmark on 11 datasets with their known grounth truth. We record the score of the pipelines on each datasets. To compute the leaderboard table, we showcase the number of wins each pipeline has over the ARIMA pipeline.

Pipeline Outperforms ARIMA
TadGAN 8
LSTM Dynamic Thresholding 8
LSTM Autoencoder 7
Dense Autoencoder 7
Azure 0

You can find the scores of each pipeline on every signal recorded in the details Google Sheets document. The summarized results can also be browsed in the following summary Google Sheets document.

Resources

Additional resources that might be of interest:

Citation

If you use Orion for your research, please consider citing the following paper:

Alexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, Kalyan Veeramachaneni. TadGAN - Time Series Anomaly Detection Using Generative Adversarial Networks.

@inproceedings{geiger2020tadgan,
  title={TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks},
  author={Geiger, Alexander and Liu, Dongyu and Alnegheimish, Sarah and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
  booktitle={2020 IEEE International Conference on Big Data (IEEE BigData)},
  organization={IEEE},
  year={2020}
}

History

0.1.5 - 2020-12-25

This version includes the new style of documentation and a revamp of the README.md. In addition to some minor improvements in the benchmark code and primitives. This release includes the transfer of tadgan pipeline to verified.

Issues resolved

  • Link with google colab - Issue #144 by @sarahmish
  • Add timeseries_anomalies unittests - Issue #136 by @sarahmish
  • Update find_sequences in converting series to arrays - Issue #135 by @sarahmish
  • Definition of error/critic smooth window in score anomalies primitive - Issue #132 by @sarahmish
  • Train-test split in benchmark enhancement - Issue #130 by @sarahmish

0.1.4 - 2020-10-16

Minor enhancements to benchmark

  • Load ground truth before try-catch - Issue #124 by @sarahmish
  • Converting timestamp to datetime in Azure primitive - Issue #123 by @sarahmish
  • Benchmark exceptions - Issue #120 by @sarahmish

0.1.3 - 2020-09-29

New benchmark and Azure primitive.

  • Implement a benchmarking function new feature - Issue #94 by @sarahmish
  • Add azure anomaly detection as primitive new feature - Issue #97 by @sarahmish
  • Critic and reconstruction error combination - Issue #99 by @sarahmish
  • Fixed threshold for find_anomalies - Issue #101 by @sarahmish
  • Add an option to have window size and window step size as percentages of error size - Issue #102 by @sarahmish
  • Organize pipelines into verified and sandbox - Issue #105 by @sarahmish
  • Ground truth parameter name enhancement - Issue #114 by @sarahmish
  • Add benchmark dataset list and parameters to s3 bucket enhancement - Issue #118 by @sarahmish

0.1.2 - 2020-07-03

New Evaluation sub-package and refactor TadGAN.

  • Two bugs when saving signalrun if there is no event detected - Issue #92 by @dyuliu 
  • File encoding/decoding issues about README.md and HISTORY.md - Issue #88 by @dyuliu
  • Fix bottle neck of score_anomaly in Cyclegan primitive - Issue #86 by @dyuliu
  • Adjust epoch meaning in Cyclegan primitive - Issue #85 by @sarahmish
  • Rename evaluation to benchmark and metrics to evaluation - Issue #83 by @sarahmish
  • Scoring function for intervals of size one - Issue #76 by @sarahmish

0.1.1 - 2020-05-11

New class and function based interfaces.

  • Implement the Orion Class - Issue #79 by @csala
  • Implement new functional interface - Issue #80 by @csala

0.1.0 - 2020-04-23

First Orion release to PyPI: https://pypi.org/project/orion-ml/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

orion-ml-0.1.6.dev0.tar.gz (1.0 MB view details)

Uploaded Source

Built Distribution

orion_ml-0.1.6.dev0-py2.py3-none-any.whl (104.2 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file orion-ml-0.1.6.dev0.tar.gz.

File metadata

  • Download URL: orion-ml-0.1.6.dev0.tar.gz
  • Upload date:
  • Size: 1.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.3.1.post20200622 requests-toolbelt/0.9.1 tqdm/4.47.0 CPython/3.6.10

File hashes

Hashes for orion-ml-0.1.6.dev0.tar.gz
Algorithm Hash digest
SHA256 146018304c95c05e51e64c01f4d85d975416121a2f03255237904a42cba104c4
MD5 53eb0c4dbf000e86dbf635a8ee658381
BLAKE2b-256 7e97529b3f7d3e1406ea6045ea5c87af4fded32a1b360df3e04c13ebf55f0fd0

See more details on using hashes here.

File details

Details for the file orion_ml-0.1.6.dev0-py2.py3-none-any.whl.

File metadata

  • Download URL: orion_ml-0.1.6.dev0-py2.py3-none-any.whl
  • Upload date:
  • Size: 104.2 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.3.1.post20200622 requests-toolbelt/0.9.1 tqdm/4.47.0 CPython/3.6.10

File hashes

Hashes for orion_ml-0.1.6.dev0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 0aab7685dfe3cbe398a81dbfe14cb2861351a286107364440197bed7c8bccba3
MD5 b652525a7d2a6cf39bf3185997bc5a8f
BLAKE2b-256 5af5e2b0961780ee955c473276d5730a19cc260e3505c30642bdfd2ecdac8602

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page