aiverify-environment-corruptions 2.0.0a1

Part of AI Verify image corruption toolbox. This package includes algorithms that add environmental corruptions (rain, fog and snow) to images at 5 severity levels, to test the robustness of machine learning models.

Algorithm - Environment Corruptions

Description

• Robustness plugin with environment corruptions

License

• Licensed under Apache Software License 2.0

Developers:

• AI Verify

Installation

Each test algorithm can now be installed via pip and run individually.

pip install aiverify-environment-corruptions==2.0.0a1

[!NOTE]
This plugin requires on imagemagick to be installed. Follow the steps below to resolve the dependency.

# install imagemagick v6
brew uninstall imagemagick
brew install imagemagick@6
brew unlink imagemagick
brew link imagemagick@6 --force

# ~/.bashrc or ~/.zshrc
export BREW_PREFIX=$(brew --prefix)
export MAGICK_HOME="$BREW_PREFIX/opt/imagemagick@6"

Example Usage:

Run the following bash script to execute the plugin

#!/bin/bash

root_path="<PATH_TO_FOLDER>/aiverify/stock-plugins/user_defined_files"

python -m aiverify_environment_corruptions \
  --data_path $root_path/data/raw_fashion_image_10 \
  --model_path $root_path/pipeline/multiclass_classification_image_mnist_fashion \
  --ground_truth_path $root_path/data/pickle_pandas_fashion_mnist_annotated_labels_10.sav \
  --ground_truth label \
  --model_type CLASSIFICATION \
  --run_pipeline \
  --annotated_ground_truth_path $root_path/data/pickle_pandas_fashion_mnist_annotated_labels_10.sav \
  --set_seed 10 \
  --file_name_label file_name

If the algorithm runs successfully, the results of the test will be saved in an output folder.

Develop plugin locally

Execute the below bash script in the project root

#!/bin/bash

# setup virtual environment
python -m venv .venv
source .venv/bin/activate

# execute plugin
cd aiverify/stock-plugins/aiverify.stock.image-corruption-toolbox/algorithms/environment_corruptions/

# install aiverify-test-engine 
pip install -e '.[dev]'

python -m aiverify_environment_corruptions --data_path  <data_path> --model_path <model_path> --ground_truth_path <ground_truth_path> --ground_truth <str> --model_type CLASSIFICATION --run_pipeline --set_seed <int> --annotated_ground_truth_path <annotated_file_path> --file_name_label <str>

Build Plugin

cd aiverify/stock-plugins/aiverify.stock.image-corruption-toolbox/algorithms/environment_corruptions/
hatch build

Tests

Pytest is used as the testing framework.

Run the following steps to execute the unit and integration tests inside the tests/ folder

cd aiverify/stock-plugins/aiverify.stock.image-corruption-toolbox/algorithms/environment_corruptions/
pytest .

Run using Docker

In the aiverify root directory, run the below command to build the docker image

docker build -t aiverify-environment-corruptions:v2.0.0a1 -f stock-plugins/aiverify.stock.image-corruption-toolbox/algorithms/environment_corruptions/Dockerfile .

Run the below bash script to run the algorithm

#!/bin/bash
docker run \
  -v $(pwd)/stock-plugins/user_defined_files:/input \
  -v $(pwd)/output:/app/aiverify/output \
  aiverify-environment-corruptions:v2.0.0a1 \
  --data_path /input/data/raw_fashion_image_10 \
  --model_path /input/pipeline/multiclass_classification_image_mnist_fashion \
  --ground_truth_path /input/data/pickle_pandas_fashion_mnist_annotated_labels_10.sav \
  --ground_truth label \
  --model_type CLASSIFICATION \
  --run_pipeline \
  --annotated_ground_truth_path /input/data/pickle_pandas_fashion_mnist_annotated_labels_10.sav \
  --set_seed 10 \
  --file_name_label file_name

If the algorithm runs successfully, the results of the test will be saved in an output folder in the working directory.

Tests

Pytest is used as the testing framework.

Run the following steps to execute the unit and integration tests inside the tests/ folder

docker run --entrypoint python3 aiverify-environment-corruptions:v2.0.0a1 -m pytest .