Hardened Extension of the Adversarial Robustness Toolbox (HEART)

HEART is a Python extension library for Machine Learning Security that builds on the popular Adversarial Robustness algorithms within the Adversarial Robustness Toolbox (ART). The extension library allows the user to leverage core ART algorithms while providing additional benefits to AI Test & Evaluation (T&E) engineers. HEART documentation can be found here.

• Support for T&E of models for Department of Defense use cases
• Alignment to MAITE protocols for seamless T&E workflows
• Essential subset of adversarial robustness methods for targeted AI security coverage
• Quality assurance of model assessments in the form of metadata
• In-depth support for users based on codified T&E expert experience in form of guides and examples
• Front-end application for low-code users: HEART Gradio Application

Installation

From Python Packaging Index (PyPI)

To install the latest version of HEART from PyPI, run:

pip install heart-library

From IBM GitHub Source

To install the latest version of HEART from the heart-library public GitHub, run:

git clone https://github.com/IBM/heart-library.git
cd heart-library
pip install .

(Optional) Development Environment via Poetry

In some cases, it may be beneficial for developers to set up an environment from a reproducible source of truth. This environment is useful for developers that wish to work within a pull request or leverage the same development conditions used by HEART contributors. Please follow the instructions for installation via Poetry within the official HEART repository:

• Poetry Installation Instructions

Getting Started With HEART

IBM has published a catalog of notebooks designed to assist developers of all skill levels with the process of getting started utilizing HEART in their AI T&E workflows. These Jupyter notebooks can be accessed within the official heart-library GitHub repository:

• HEART Jupyter Notebooks

HEART Modules

The HEART library is organized into three primary modules: attacks, estimators, and metrics.

heart_library.attacks

The HEART attacks module contains implementations of attack algorithms for generating adversarial examples and evaluating model robustness.

heart_library.estimators

The HEART estimators module contains classes that wrap and extend the evaluated model to make it compatible with attacks and metrics.

heart_library.metrics

The HEART metrics module implements industry standard, commonly-used T&E metrics for model evaluation.

Acknowledgement

This material is based upon work supported by the Chief Digital and Artificial Intelligence Office under Contract No. W519TC-23-9-2037. The views and conclusions contained herein are those of the author(s) and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Government.