Skip to main content

IBM Analog Hardware Acceleration Kit

Project description

IBM Analog Hardware Acceleration Kit

PyPI test-and-lint release-build Documentation Status PyPI - License arXiv

Description

IBM Analog Hardware Acceleration Kit is an open source Python toolkit for exploring and using the capabilities of in-memory computing devices in the context of artificial intelligence.

:warning: This library is currently in beta and under active development. Please be mindful of potential issues and keep an eye for improvements, new features and bug fixes in upcoming versions.

The toolkit consists of two main components:

Pytorch integration

A series of primitives and features that allow using the toolkit within PyTorch:

  • Analog neural network modules (fully connected layer, 1d/2d/3d convolution layers, LSTM layer, sequential container).
  • Analog training using torch training workflow:
    • Analog torch optimizers (SGD).
    • Analog in-situ training using customizable device models and algorithms (Tiki-Taka).
  • Analog inference using torch inference workflow:
    • State-of-the-art statistical model of a phase-change memory (PCM) array calibrated on hardware measurements from a 1 million PCM devices chip.
    • Hardware-aware training with hardware non-idealities and noise included in the forward pass to make the trained models more robust during inference on Analog hardware.

Analog devices simulator

A high-performant (CUDA-capable) C++ simulator that allows for simulating a wide range of analog devices and crossbar configurations by using abstract functional models of material characteristics with adjustable parameters. Features include:

  • Forward pass output-referred noise and device fluctuations, as well as adjustable ADC and DAC discretization and bounds
  • Stochastic update pulse trains for rows and columns with finite weight update size per pulse coincidence
  • Device-to-device systematic variations, cycle-to-cycle noise and adjustable asymmetry during analog update
  • Adjustable device behavior for exploration of material specifications for training and inference
  • State-of-the-art dynamic input scaling, bound management, and update management schemes

Other features

Along with the two main components, the toolkit includes other functionalities such as:

  • A library of device presets that are calibrated to real hardware data and based on models in the literature, along with a configuration that specifies a particular device and optimizer choice.
  • A module for executing high-level use cases ("experiments"), such as neural network training with minimal code overhead.
  • A utility to automatically convert a downloaded model (e.g., pre-trained) to its equivalent Analog model by replacing all linear/conv layers to Analog layers (e.g., for convenient hardware-aware training).
  • Integration with the AIHW Composer platform, a no-code web experience that allows executing experiments in the cloud.

How to cite?

In case you are using the IBM Analog Hardware Acceleration Kit for your research, please cite the AICAS21 paper that describes the toolkit:

Malte J. Rasch, Diego Moreda, Tayfun Gokmen, Manuel Le Gallo, Fabio Carta, Cindy Goldberg, Kaoutar El Maghraoui, Abu Sebastian, Vijay Narayanan. "A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays" (2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems)

Usage

Training example

from torch import Tensor
from torch.nn.functional import mse_loss

# Import the aihwkit constructs.
from aihwkit.nn import AnalogLinear
from aihwkit.optim import AnalogSGD

x = Tensor([[0.1, 0.2, 0.4, 0.3], [0.2, 0.1, 0.1, 0.3]])
y = Tensor([[1.0, 0.5], [0.7, 0.3]])

# Define a network using a single Analog layer.
model = AnalogLinear(4, 2)

# Use the analog-aware stochastic gradient descent optimizer.
opt = AnalogSGD(model.parameters(), lr=0.1)
opt.regroup_param_groups(model)

# Train the network.
for epoch in range(10):
    pred = model(x)
    loss = mse_loss(pred, y)
    loss.backward()

    opt.step()
    print('Loss error: {:.16f}'.format(loss))

You can find more examples in the examples/ folder of the project, and more information about the library in the documentation. Please note that the examples have some additional dependencies - you can install them via pip install -r requirements-examples.txt. You can find interactive notebooks and tutorials in the notebooks/ directory.

Further reading

We also recommend to take a look at the tutorial article that describes the usage of the toolkit that can be found here:

Manuel Le Gallo, Corey Lammie, Julian Buechel, Fabio Carta, Omobayode Fagbohungbe, Charles Mackin, Hsinyu Tsai, Vijay Narayanan, Abu Sebastian, Kaoutar El Maghraoui, Malte J. Rasch. "Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference" (APL Machine Learning Journal:1(4) 2023)

What is Analog AI?

In traditional hardware architecture, computation and memory are siloed in different locations. Information is moved back and forth between computation and memory units every time an operation is performed, creating a limitation called the von Neumann bottleneck.

Analog AI delivers radical performance improvements by combining compute and memory in a single device, eliminating the von Neumann bottleneck. By leveraging the physical properties of memory devices, computation happens at the same place where the data is stored. Such in-memory computing hardware increases the speed and energy efficiency needed for next-generation AI workloads.

What is an in-memory computing chip?

An in-memory computing chip typically consists of multiple arrays of memory devices that communicate with each other. Many types of memory devices such as phase-change memory (PCM), resistive random-access memory (RRAM), and Flash memory can be used for in-memory computing.

Memory devices have the ability to store synaptic weights in their analog charge (Flash) or conductance (PCM, RRAM) state. When these devices are arranged in a crossbar configuration, it allows to perform an analog matrix-vector multiplication in a single time step, exploiting the advantages of analog storage capability and Kirchhoff’s circuits laws. You can learn more about it in our online demo.

In deep learning, data propagation through multiple layers of a neural network involves a sequence of matrix multiplications, as each layer can be represented as a matrix of synaptic weights. The devices are arranged in multiple crossbar arrays, creating an artificial neural network where all matrix multiplications are performed in-place in an analog manner. This structure allows to run deep learning models at reduced energy consumption.

Awards and Media Mentions

Installation

Installing from PyPI

The preferred way to install this package is by using the Python package index:

pip install aihwkit

Conda-based Installation

There is a conda package for aihwkit available in conda-forge. It can be installed in a conda environment running on a Linux or WSL in a Windows system.

  • CPU

    conda install -c conda-forge aihwkit
    
  • GPU

    conda install -c conda-forge aihwkit-gpu
    

If you encounter any issues during download or want to compile the package for your environment, please take a look at the advanced installation guide. That section describes the additional libraries and tools required for compiling the sources using a build system based on cmake.

Docker Installation

For GPU support, you can also build a docker container following the CUDA Dockerfile instructions. You can then run a GPU enabled docker container using the follwing command from your peoject dircetory

docker run --rm -it --gpus all -v $(pwd):$HOME --name aihwkit aihwkit:cuda bash

Authors

IBM Research has developed IBM Analog Hardware Acceleration Kit, with Malte Rasch, Diego Moreda, Fabio Carta, Julian Büchel, Corey Lammie, Charles Mackin, Kim Tran, Tayfun Gokmen, Manuel Le Gallo-Bourdeau, and Kaoutar El Maghraoui as the initial core authors, along with many contributors.

You can contact us by opening a new issue in the repository or alternatively at the aihwkit@us.ibm.com email address.

License

This project is licensed under [MIT License].

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aihwkit-1.0.0.tar.gz (639.5 kB view details)

Uploaded Source

Built Distributions

aihwkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

aihwkit-1.0.0-cp310-cp310-macosx_15_0_arm64.whl (10.0 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

aihwkit-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl (758.4 kB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

aihwkit-1.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.4 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

aihwkit-1.0.0-cp39-cp39-macosx_15_0_arm64.whl (10.0 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

aihwkit-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl (749.6 kB view details)

Uploaded CPython 3.9macOS 10.9+ x86-64

aihwkit-1.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.4 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.17+ x86-64

aihwkit-1.0.0-cp38-cp38-macosx_15_0_arm64.whl (10.0 MB view details)

Uploaded CPython 3.8macOS 15.0+ ARM64

aihwkit-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl (748.9 kB view details)

Uploaded CPython 3.8macOS 10.9+ x86-64

File details

Details for the file aihwkit-1.0.0.tar.gz.

File metadata

  • Download URL: aihwkit-1.0.0.tar.gz
  • Upload date:
  • Size: 639.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.16

File hashes

Hashes for aihwkit-1.0.0.tar.gz
Algorithm Hash digest
SHA256 129799705db6031dda5dfae9f0fea92db944fa7f8d92ff518ae287cfb66b306b
MD5 1fe674d3ceae9bb0736b59b854ee1c0b
BLAKE2b-256 157b2e5f1d798842a44425d5ba2e95c238288c1b4e39d0b54df2255272a821a5

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7c63da48c5c0c9c5fbfa16357a287c7c818a86d20e7e65a2cc6b51be29cd1421
MD5 709e33e63660ec8dd28354c8ee36ba2e
BLAKE2b-256 2ac343b3f81a278f8db9290360d92592c39e93df3bb6473b2aa12558fb1b49a0

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 f21b93caef32c5528b8b711c0bf489fd60338f07c8339f1085428a7c88024c94
MD5 3cefc07a88324f5acc0fbb3a8e4419bd
BLAKE2b-256 0b62f17caf730d3ba0b1096fa806e2b1124694ad612e546c7362eb99158e8b9e

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 99427948941fe10c516b12b65a5c2831837389aca62778f9921618fd4a20728b
MD5 425134301fdae064bd9cdd9eae4e0e8b
BLAKE2b-256 73c6dff2514db49f970b0d5cccff272481ef1e1e0ab30816925b9bb8fb72bc90

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 494de0aeae30ee414bdc5c47ebee2fc6477b369173c7feb7aa56737ceb34aeb6
MD5 497fcd86d5b848abce9088f7436bbc92
BLAKE2b-256 1a19446ced977ff79c1872da65ddaec4e176fa53d4c61469c2c216a6f7abc117

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 86903ff7ea1c530f868a084dc6a083e2f7a16a20f063e82234a93c56d4acd4ef
MD5 1fdc3b9ec90244f2ff3cfc9e04af534f
BLAKE2b-256 c9b88d1caa7151fbe27f6539f0704ae6daa4a1bc423fd5c85ac2e878ddaad61c

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 39000e10e26f2c2239671376549f1592ecffabe124e7307d0097c635b261b41d
MD5 34d08ee2b316b9f82f2d7b652d393a1a
BLAKE2b-256 6a5e97e795adfd0ef610280d7ace547cdae94b62b8fd8e9aa57ba1efe5f210aa

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c8b4867d1a4d62b9ec803ae917efccc6cb2a445b2b03954d684cd9f3d7e4c40a
MD5 f65df0dec9a942a101c2d11773fe0fbd
BLAKE2b-256 ec87af9b6dc57438d15ec26c6646080abf537972a470cf1993a78677301fb84c

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp38-cp38-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp38-cp38-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 c1390d6fab4870585d1004b22c349a4909aa68f3484b486e1d847bedd74b521a
MD5 dc3a3f610c8951782b6260d58611eceb
BLAKE2b-256 aa329d273b85071406d048bdce5f02169d574eed1b5c0e1fa1ae952eb1a02d9d

See more details on using hashes here.

File details

Details for the file aihwkit-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for aihwkit-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 ff7ff9dca619701d296a14625215e0c69dd3d3b47bdb2e8ad31d07844bcbd5ed
MD5 c735f21a6275cc14914d7f6e88a122f9
BLAKE2b-256 a453c2cb7920090aef9e05319c388b9097f5bf130c13e15244c1cceb2d0ab813

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page