Anomaly Detection Framework allows us to calculate Anomalities on any Time - Series Data Sets. It has an interface which is easy to manage to train - predict with given dataset.
Project description
Anomaly Detection Framework
Anomaly-Detection-Framework is a platform for Time Series Anomaly Detection Proplems. Give the data to platform get the Anomaly Labels with scheduled time perioeds. That is simple is that!!! Anomaly-Detection-Framework unables to Data Science communities easy to detect abnormal values on a Time Series Data Set. It is a platform which can run on Docker containers as services or python by using its modules. It also has the web interface which allows us to train - prediction - parameter tuning jobs easly.
Key Features
-
Web Interface:
It is web interface which allows us to connect data source and execute Machine Learning processes. You may create tasks according to your data set. These tasks are train models, Anomaly Detection Prediction, Parameter Tunning For Train Models.
-
Schedule Machine Learning Jobs:
Three main process are able to initialized from platform Train, prediction and Parameter Tuning. Each Process can be scheduled Daily, Monthly, Weekly Hourly, etc with given time. In addition to that, you may run your Machine Learning processes real time.
-
Dashboard Visualization:
When your data have connec to a data source and assign Date Indicator, Model Dimensions (optional), and feature column from Create Task Menu Create Task, you may see the dashboard from Dashboard Menu.
-
Data Sources: Here are the data source you can conect with your SQL queries:
- Ms SQL Server
- PostgreSQL
- AWS RedShift
- Google BigQuery
- .csv
- .json
- pickle
-
Models:
There are 2 Anomaly Detection Algorithm and FBProphet Anomaly Detection Solution are running on the platform.
- LSTM
- FBProphet
- Isolation Forest
-
API Services:
There are 4 Services run on the platform.
- Machine Learning Schedule Services
- LSTM Model Service
- FbProphet Model Service
- Isolation Foreset model Service
-
Docker Compose Integration (Beta):
Threse 4 containers are running on containers as services.
- ml_executor-services
- model-services-iso_f
- model-services-lstm
- model-services-prophet
Running Platform
1. You Have to Specify you directory
from anomaly_detection import ad_execute as ad_exec
ad = ad_exec.AnomalyDetection(path='./Desktop', environment='local')
ad.init(apis=None)
Once, you have assigned the path a folder called 'Anomaly_Detection_Framework' will be created. Trained models will be imported to models file. Logs for both 'ml_execute', 'model_iso_f, 'model_prophet' and 'model_lstm' of log files will be created at logs file. Your .csv, .json or.yaml data source file must be copied to the data folder which is at 'Anomaly_Detection_Framework' folder.
-
anomaly_detection.AnomalyDetection:
-
AnomalyDetection
path : Location where you are willing to create models and prediction data set.
*enviroment : local or docker
host : local or docker
remove_existed : remove data from the location where you have entered as path.
master_node : if False, you must enter the services of information manually (port host, etc.). This allows user to initialize each service on different location or server or worker. If master_node=False there will not be a web interface. Once you create a master node, in order to use these services, you have clerify these services on a it.
-
init
This initialized the folders. Checks the available ports for services in range between 6000 - 7000. Updates the apis.yaml if it is necessary.
apis :
{ 'model_iso_f': {'port': 6000, 'host': '127.0.0.1'}, 'model_lstm': {'port': 6001, 'host': '127.0.0.1'} }
-
2. Run The PLatform
ad.run_platform()
This process initializesthe platform. Once you have run the code below you may have seen the services are running. If you assign master_node=True you may use enter to web interface from http://127.0.0.1:7002/. If 7002 port is used from another platform directly platform assgns +1 port. (7003,7004, 7005, ..)
2. Data Source
You can connecto to data source from Data Source Configuraitons. There is two option to connect to a data source. You can integrate on web interface or you can use AnomalyDetection method in order to able to connect a data source.
- Connection PostgreSQL - MS SQL
- Connection .csv - .json - .yaml
- Connection Google BigQuery
- Create Tasks After Data Source is created
- Job Run
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for anomaly_detection_framework-0.0.11.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9daa193de7e70f30556aed630f27d196f46fccf519686d6a672a6414677c65b1 |
|
MD5 | 4bde468491aa01aee35a6dd672eb6d0e |
|
BLAKE2b-256 | 330ea8993cc44e2373f5d7fa804664604af8706066d2d3d6338ed8f4cce7e980 |
Hashes for anomaly_detection_framework-0.0.11-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 232f3e7f31c9038345c288cd0d1b2b66bf82095727e90d272a63e9c2fa34812a |
|
MD5 | e4eaf95423b1e0fea84d027b63f2aa04 |
|
BLAKE2b-256 | 9cc783182989c71ea584dfbaf588c7da4d00a5fc91f13bc0a83295236943eb7e |