Skip to main content

IBM Analog Hardware Acceleration Kit

Project description

IBM Analog Hardware Acceleration Kit

PyPI Documentation Status Build Status PyPI - License arXiv

Description

IBM Analog Hardware Acceleration Kit is an open source Python toolkit for exploring and using the capabilities of in-memory computing devices in the context of artificial intelligence.

:warning: This library is currently in beta and under active development. Please be mindful of potential issues and keep an eye for improvements, new features and bug fixes in upcoming versions.

The toolkit consists of two main components:

Pytorch integration

A series of primitives and features that allow using the toolkit within PyTorch:

  • Analog neural network modules (fully connected layer, 1d/2d/3d convolution layers, LSTM layer, sequential container).
  • Analog training using torch training workflow:
    • Analog torch optimizers (SGD).
    • Analog in-situ training using customizable device models and algorithms (Tiki-Taka).
  • Analog inference using torch inference workflow:
    • State-of-the-art statistical model of a phase-change memory (PCM) array calibrated on hardware measurements from a 1 million PCM devices chip.
    • Hardware-aware training with hardware non-idealities and noise included in the forward pass to make the trained models more robust during inference on Analog hardware.

Analog devices simulator

A high-performant (CUDA-capable) C++ simulator that allows for simulating a wide range of analog devices and crossbar configurations by using abstract functional models of material characteristics with adjustable parameters. Features include:

  • Forward pass output-referred noise and device fluctuations, as well as adjustable ADC and DAC discretization and bounds
  • Stochastic update pulse trains for rows and columns with finite weight update size per pulse coincidence
  • Device-to-device systematic variations, cycle-to-cycle noise and adjustable asymmetry during analog update
  • Adjustable device behavior for exploration of material specifications for training and inference
  • State-of-the-art dynamic input scaling, bound management, and update management schemes

Other features

Along with the two main components, the toolkit includes other functionalities such as:

  • A library of device presets that are calibrated to real hardware data and based on models in the literature, along with a configuration that specifies a particular device and optimizer choice.
  • A module for executing high-level use cases ("experiments"), such as neural network training with minimal code overhead.
  • A utility to automatically convert a downloaded model (e.g., pre-trained) to its equivalent Analog model by replacing all linear/conv layers to Analog layers (e.g., for convenient hardware-aware training).
  • Integration with the AIHW Composer platform, a no-code web experience that allows executing experiments in the cloud.

How to cite?

In case you are using the IBM Analog Hardware Acceleration Kit for your research, please cite the AICAS21 paper that describes the toolkit:

Malte J. Rasch, Diego Moreda, Tayfun Gokmen, Manuel Le Gallo, Fabio Carta, Cindy Goldberg, Kaoutar El Maghraoui, Abu Sebastian, Vijay Narayanan. "A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays" (2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems)

Usage

Training example

from torch import Tensor
from torch.nn.functional import mse_loss

# Import the aihwkit constructs.
from aihwkit.nn import AnalogLinear
from aihwkit.optim import AnalogSGD

x = Tensor([[0.1, 0.2, 0.4, 0.3], [0.2, 0.1, 0.1, 0.3]])
y = Tensor([[1.0, 0.5], [0.7, 0.3]])

# Define a network using a single Analog layer.
model = AnalogLinear(4, 2)

# Use the analog-aware stochastic gradient descent optimizer.
opt = AnalogSGD(model.parameters(), lr=0.1)
opt.regroup_param_groups(model)

# Train the network.
for epoch in range(10):
    pred = model(x)
    loss = mse_loss(pred, y)
    loss.backward()

    opt.step()
    print('Loss error: {:.16f}'.format(loss))

You can find more examples in the examples/ folder of the project, and more information about the library in the documentation. Please note that the examples have some additional dependencies - you can install them via pip install -r requirements-examples.txt. You can find interactive notebooks and tutorials in the notebooks/ directory.

Further reading

We also recommend to take a look at the tutorial article that describes the usage of the toolkit that can be found here:

Manuel Le Gallo, Corey Lammie, Julian Buechel, Fabio Carta, Omobayode Fagbohungbe, Charles Mackin, Hsinyu Tsai, Vijay Narayanan, Abu Sebastian, Kaoutar El Maghraoui, Malte J. Rasch. "Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference" (APL Machine Learning Journal:1(4) 2023)

What is Analog AI?

In traditional hardware architecture, computation and memory are siloed in different locations. Information is moved back and forth between computation and memory units every time an operation is performed, creating a limitation called the von Neumann bottleneck.

Analog AI delivers radical performance improvements by combining compute and memory in a single device, eliminating the von Neumann bottleneck. By leveraging the physical properties of memory devices, computation happens at the same place where the data is stored. Such in-memory computing hardware increases the speed and energy efficiency needed for next-generation AI workloads.

What is an in-memory computing chip?

An in-memory computing chip typically consists of multiple arrays of memory devices that communicate with each other. Many types of memory devices such as phase-change memory (PCM), resistive random-access memory (RRAM), and Flash memory can be used for in-memory computing.

Memory devices have the ability to store synaptic weights in their analog charge (Flash) or conductance (PCM, RRAM) state. When these devices are arranged in a crossbar configuration, it allows to perform an analog matrix-vector multiplication in a single time step, exploiting the advantages of analog storage capability and Kirchhoff’s circuits laws. You can learn more about it in our online demo.

In deep learning, data propagation through multiple layers of a neural network involves a sequence of matrix multiplications, as each layer can be represented as a matrix of synaptic weights. The devices are arranged in multiple crossbar arrays, creating an artificial neural network where all matrix multiplications are performed in-place in an analog manner. This structure allows to run deep learning models at reduced energy consumption.

Awards and Media Mentions

Installation

Installing from PyPI

The preferred way to install this package is by using the Python package index:

pip install aihwkit

Conda-based Installation

There is a conda package for aihwkit available in conda-forge. It can be installed in a conda environment running on a Linux or WSL in a Windows system.

  • CPU

    conda install -c conda-forge aihwkit
    
  • GPU

    conda install -c conda-forge aihwkit-gpu
    

If you encounter any issues during download or want to compile the package for your environment, please take a look at the advanced installation guide. That section describes the additional libraries and tools required for compiling the sources using a build system based on cmake.

Docker Installation

For GPU support, you can also build a docker container following the CUDA Dockerfile instructions. You can then run a GPU enabled docker container using the follwing command from your peoject dircetory

docker run --rm -it --gpus all -v $(pwd):$HOME --name aihwkit aihwkit:cuda bash

Authors

IBM Research has developed IBM Analog Hardware Acceleration Kit, with Malte Rasch, Diego Moreda, Fabio Carta, Julian Büchel, Corey Lammie, Charles Mackin, Kim Tran, Tayfun Gokmen, Manuel Le Gallo-Bourdeau, and Kaoutar El Maghraoui as the initial core authors, along with many contributors.

You can contact us by opening a new issue in the repository or alternatively at the aihwkit@us.ibm.com email address.

License

This project is licensed under Apache License 2.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aihwkit-0.9.2.tar.gz (612.5 kB view details)

Uploaded Source

Built Distributions

aihwkit-0.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

aihwkit-0.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

aihwkit-0.9.2-cp39-cp39-macosx_10_9_x86_64.whl (10.7 MB view details)

Uploaded CPython 3.9 macOS 10.9+ x86-64

aihwkit-0.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

aihwkit-0.9.2-cp38-cp38-macosx_10_9_x86_64.whl (10.7 MB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

File details

Details for the file aihwkit-0.9.2.tar.gz.

File metadata

  • Download URL: aihwkit-0.9.2.tar.gz
  • Upload date:
  • Size: 612.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.14

File hashes

Hashes for aihwkit-0.9.2.tar.gz
Algorithm Hash digest
SHA256 cbcc4410830786edb510564dcbdc4c289490b9099d43535ba71bb8de999fef26
MD5 361b706d8585033586f0797dc4068040
BLAKE2b-256 7bed385adb141f3691000c33adb3d76c80e1fe213d3e991b91054066e4f698b6

See more details on using hashes here.

File details

Details for the file aihwkit-0.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-0.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 15b997086f7a62213bccdfe06c8506cf8588ad7aebc44a572a84a3e8e1ea7395
MD5 fa6eb8a488124f0bdeac6f86b3b82ee3
BLAKE2b-256 33c5680f19cca0b33e648e42627f4c168806c367fe9254bc2c97db2b024ddda8

See more details on using hashes here.

File details

Details for the file aihwkit-0.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-0.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0e186a4758720144404c9975ca217ff631f08855f6ec80c9e70aa36a79ae611c
MD5 a565e7c154604c34579dfccd258a9069
BLAKE2b-256 bb644f93ca618590bc773ae5adf8511c950b1733781bdcb18c49ef5db24b5a0e

See more details on using hashes here.

File details

Details for the file aihwkit-0.9.2-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-0.9.2-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 2131af4f713a699dd4c58e9fdb720ac09b1649a78cf4fe33d73b9d59e21eb9af
MD5 a364fd4c5fdac5a6c4f2e48dea8ae6f7
BLAKE2b-256 744672a05502f14f4ddf7a841a645f528d6fe5acb509139680ee3d5facfab4f6

See more details on using hashes here.

File details

Details for the file aihwkit-0.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-0.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ed5fedb8f90961607e55f81609f85a8673190b5644d968b5cf068eb60639d6fe
MD5 d58f287c98f72c8d2b6114dd850b3b3d
BLAKE2b-256 38bffd6451c6cc50b5b8d005ec074af7e0eddf7a461fd1859cf3e64a51547156

See more details on using hashes here.

File details

Details for the file aihwkit-0.9.2-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-0.9.2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 8c1e2f4202afa307019dd82a4bf0d3877a57c933b38a36b3cbc83f37094e5fe5
MD5 f1b722602296e0d450e5c181efb070ae
BLAKE2b-256 6600bfb1caa379227f2cab057c20405f50a451d665992084619d43d59a9aedaa

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page