amuletml 0.1.0

Amulet is a Python machine learning (ML) package to evaluate the susceptibility of different risks to security, privacy, and fairness.

Amulet

Amulet is a Python machine learning (ML) package to evaluate the susceptibility of different risks to security, privacy, and fairness. Amulet is applicable to evaluate how algorithms designed to reduce one risk may impact another unrelated risk and compare different attacks/defenses for a given risk.

Amulet builds upon prior work titled “SoK: Unintended Interactions among Machine Learning Defenses and Risks” which appears in IEEE Symposium on Security and Privacy 2024. The SoK covers only two interactions and identifies the design of a software library to evaluate unintended interactions as future work. Amulet addresses this gap by including eight different risks each covering their own attacks, defenses and metrics.

Amulet is:

• Comprehensive: Covers the most representative attacks/defenses/metrics for different risks.
• Extensible: Easy to include additional risks, attacks, defenses, or metrics.
• Consistent: Allows using different attacks/defenses/metrics with a consistent, easy-to-use API.
• Applicable: Allows evaluating unintended interactions among defenses and attacks.

Built to work with PyTorch, you can incorporate Amulet into your current ML pipeline to test how your model interacts with these state-of-the-art defenses and risks. Alternatively, you can use the example pipelines to bootstrap your pipeline.

Requirements

Note: The package requires the CUDA version to be 11.8 or above for PyTorch 2.2

Install poetry

python3 -m venv .poetry_venv

. .poetry_venv/bin/activate or . .venv/bin/activate.fish

python -m pip install --upgrade pip

pip install poetry

deactivate

Consider setting .poetry_venv/bin/poetry config virtualenvs.create false to prevent poetry from creating its own venv.

Main venv

To create the virtual environemnt: python3 -m venv .venv

To activate it: source .venv/bin/activate or if using fish . .venv/bin/activate.fish

Then, to install the dependencies: .poetry_venv/bin/poetry install

DISCLAIMER: Installing pytorch with poetry is still weird but should work.

Using poetry

(Inside your .venv); when you add or modify any dependencies in pyproject.toml, run .poetry_venv/bin/poetry lock --no-update to rebuild the dependency graph. Then run .poetry_venv/bin/poetry install to install the dependencies.

pre-commit

There're some pre-commit hooks configured for this project. Also, poetry installs pre-commit as a dev dependency.

Run pre-commit install for consistent development.

Getting Started

Test installation

To test your installation, please run amulet/examples/get_started.py. This script also serves as a starting point to learn how to use the library.

Learn More

For more information on the basics about the library, please see the Getting Started guide.

To see the attacks, defenses, and risks (modules) that Amulet implements, please refer to the Module Heirarchy (link TBD) in the Tutorial (link TBD).

For each module, please see amulet/examples for implementations of pipelines that include recommendations on how to run each module.