e2eAIOK-ModelAdapter 1.0.1b2023042400

Intel® End-to-End AI Optimization Kit

Intel® End-to-End AI Optimization Kit

INTRODUCTION

Problem Statement

Modern End to End AI pipeline life cycle is quite complicate with a complex pipeline including data processing, feature engineering, model development, and model deployment & maintenance. The iterative nature for feature engineering, model testing and hyper-parameter optimization makes the process more time-consuming. This complexity creates an entry-barrier for novice and citizen data scientists who might not have such expertise or skills. Meanwhile, people tend to develop larger and larger models to get better performance, which are quite often over-parameterized. Those overparameterized models not only poses significant challenges on AI hardware infrastructure as they require expensive computation power for training, but also posed a challenge when try to deploy in resource constraint environment which is a common need.

Solution with Intel® End-to-End AI Optimization Kit

Intel® End-to-End AI Optimization Kit is a composable toolkits for E2E AI optimization to deliver high performance lightweight networks/models efficiently on commodity HW like CPU, intending to make E2E AI pipelines faster, easier and more accessible.

Making AI Faster: It reduces E2E time on CPU to an acceptable range throughput full pipeline optimization and improved scale-up/out capability on Intel platforms with Intel optimized framework and toolkits, delivers popular lighter DL Models with close enough performance and significantly higher inference throughput.

Making AI Easier: It automates provides simplified toolkits for data processing, distributed training, and compact neural network construction, automates E2E AI pipeline with click to run workflows and can be easily plugged to third party ML solutions/platforms as an independent composable component.

Making AI more accessible: Through built-in optimized, parameterized models generated by smart democratization advisor and domain-specific, neural architected search (NAS) based network constructure, it brings complex DL to commodity HW, everyone can easily access AI on existing CPU clusters without the need to be an expert on data engineering and data science.

This solution is intended for

This solution is intended for citizen data scientists, enterprise users, independent software vendor and partial of cloud service provider.

Papers and Blogs

• ICYMI – SigOpt Summit Recap Democratizing End-to-End Recommendation Systems
• The SigOpt Intelligent Experimentation Platform
• SDC2022 - Data Platform for End-to-end AI Democratization
• SIHG4SR: Side Information Heterogeneous Graph for Session Recommender

ARCHITECTURE

Intel® End-to-End AI Optimization Kit

Intel® End-to-End AI Optimization Kit is a composable toolkits for E2E AI optimization to deliver high performance lightweight networks/models efficiently on commodity HW. It is a pipeline framework that streamlines AI optimization technologies in each stage of E2E AI pipeline, including data processing, feature engineering, training, hyper-parameter tunning, and inference. Intel® End-to-End AI Optimization Kit delivers high performance, lightweight models efficiently on commodity hardware.

The key components are

• RecDP: A parallel data processing and feature engineering lib on top of Spark, and extensible to other data processing tools. It provides abstraction API to hide Spark programming complexity, delivers optimized performance through adaptive query plan & strategy, supports critical feature engineering functions on Tabular dataset, and can be easily integrated to third party solutions.

• Smart Democratization Advisor (SDA): A user-guided tool to facilitate automation of built-in model democratization via parameterized models, it generates yaml files based on user choice, provided build-in intelligence through parameterized models and leverage SigOpt for HPO. SDA converts the manual model tuning and optimization to assisted autoML and autoHPO. SDA provides a list of build-in optimized models ranging from RecSys, CV, NLP, ASR and RL.

• Neural Network Constructor: A neural architecture search technology and transfer learning based component to build compact neural network models for specific domains directly. It includes two componments, DE-NAS and Model Adapter. DE-NAS is a multi-model, hardware aware, train-free neural architecture search approach to build models for CV, NLP, ASR directly. Model Adapter leverages transfer learning model adaptor to deploy the models in user’s production environment.

For more information, you may read the docs.

Getting Started

Installing

Install with Baremetal Environment

• To install all components:

  • To install e2eAIOK in baremetal environment, use pip install e2eAIOK
  • To install latest nightly build, use pip install e2eAIOK --pre

• To install each individual component:

  • To install SDA, use pip install e2eAIOK-sda
  • To install DE-NAS, use pip install e2eAIOK-denas
  • To install Model Adapter, use pip install e2eAIOK-ModelAdapter

Install with Docker Environment

git clone https://github.com/intel/e2eAIOK.git
cd e2eAIOK
git submodule update --init --recursive
python scripts/start_e2eaiok_docker.py --backend [tensorflow, pytorch, pytorch112] --dataset_path ../ --workers host1, host2, host3, host4 --proxy "http://addr:ip"

Demos

• Built-in Models

  • DLRM - RecSys, PyTorch
  • DIEN - RecSys, TensorFlow
  • WND - RecSys, TensorFlow
  • RNNT - Speech Recognition, PyTorch
  • RESNET - Computer vision, TensorFlow
  • BERT - Natual Language Processing, TensorFlow
  • MiniGo - minimalist engine modeled after AlphaGo Zero, TensorFlow

• Neural network constructor

  • DE-NAS demos:

    • DE-NAS Overview
      • CNN - Computer Vision, PyTorch
      • ViT - Computer Vision, PyTorch
      • BERT - NLP, PyTorch
      • ASR - Speech Recognition, PyTorch
      • BERT - Hugging Face models, PyTorch

  • Model Aadapter demos

    • Model Adapter Overview
      • Finetuner - Computer Vision, Image Classification, ResNet50, PyTorch
      • Distiller - Computer Vision, Image Classification, ResNet18, PyTorch
      • Domain Adapter - Computer Vision, Medical Segmentation, 3D Unet, PyTorch

Performance

Performance results are evaluated on 4-node cluster configured with Intel(R) Xeon(R) Platinum 8358 Scalable processor. For DeNAS CNN and ViT, Intel® End-to-End AI Optimization Kit delivered 40.73x and 35.63x search time speedup, 82.57x and 4.44x training time speedup over ZenNAS and AutoFormer respectively. For DeNAS searched CNN, ViT, BERT and ASR model, Intel® End-to-End AI Optimization Kit delivered 9.86x, 4.44x, 7.68x and 59.12x training time speedup with 0.03x, 1.20x, 0.62x and 0.81x model size respectively. Please refer to DeNAS link for detailed test dataset and test method.

Noted: Optimized lighter models' accuracy are slightly lower: CNN -3% accuracy, ViT -5% accuracy, BERT -4% F1 score.

Performance results are evaluated on 4-node cluster configured with Intel(R) Xeon(R) Platinum 8358 Scalable processor. For MiniGO, BERT, ResNet, RNN-T, Intel® End-to-End AI Optimization Kit delivered 13.06x, 10.10x, 8.77x and 14.19x training time speedup respecitvely through E2E optimizations. Please refer to corresponding model link for detailed test dataset and test method.

Noted: Optimized lighter models' accuracy are slightly lower: ResNet -5% accuracy, BERT -1% F1 score.

Performance results are evaluated on 4-node cluster configured with Intel(R) Xeon(R) Platinum 8358 Scalable processor. For WnD, DIEN and DLRM, Intel® End-to-End AI Optimization Kit delivered 51.01x(5.02x ELT & 113.03x training), 12.67x(14.86x ELT & 11.91x training) and 71.16x(86.40x ELT & 42.31x training) E2E time speedup, 21.18x, 14.11x and 124.98x inference throughput speedup respectively. Please refer to corresponding model link for detailed test dataset and test method.

Getting Support

• Github Issues