irisml 0.0.25

Simple ML pipeline platform

IrisML

Proof of Concept for a simple framework to create a ML pipeline.

Features

• Run a ML training/inference with a simple JSON configuration.
• Modularized interfaces for task components.
• Cache task outputs for faster experiments.

Getting started

Installation

Prerequisite: python 3.8+

# Install the core framework and standard tasks.
pip install irisml irisml-tasks irisml-tasks-training

Run an example job

# Install additional packages that are required for the example
pip install irisml-tasks-torchvision

# Run on local machine
irisml_run docs/examples/mobilenetv2_mnist_training.json

Available commands

# Run the specified pipeline. You can provide environment variables by "-e" option, which will be acceible through $env variable in the json config.
irisml_run <pipeline_json_path> [-e <ENV_NAME>=<env_value>] [--no_cache] [--no_cache_read] [-v]

# Show information about the specified task. If <task_name> is not provided, shows a list of available tasks in the current environment.
irisml_show [<task_name>]

# Manage a cache storage on Azure Blob Storage
# list - Show a list of matched blobs.
# download - Download matched blobs.
# remove - Remove matched blobs.
# show - Show the contents of matched blobs.
irisml_cache <list|download|remove|show> [--mtime <+|->N] [--name NAME]

Pipeline definition

PipelineDefinition = {"tasks": List[TaskDefinition], "on_error": Optional[List[TaskDescription]]}

TaskDefinition = {
    "task": <task module name>,
    "name": <optional unique name of the task>,
    "inputs": <list of input objects>,
    "config": <config for the task. Use irisml_show command to find the available configurations.>
}

In the TaskDefinition.inputs and TaskDefinition.config, you cna use the following two variable.

• $env.<variable_name> This variable will be replaced by the environment variable that was provided as arguments for irisml_run command.
• $outputs.<task_name>.<field_name> This variable will be replaced by the outputs of the specified previous task.

It raises an exception on runtime if the specified variable was not found.

If a task raised an exception, the tasks specified in on_error field will be executed. The exception object will be assigned to "$env.IRISML_EXCEPTION" variable.

Pipeline cache

Using cache, you can modify and re-run a pipeline config with minimum cost. If the cache is enabled, IrisML will calculate hash values for all task inputs/configs and upload the task outputs to a specified storage. When it found a task with same hash values, it can download the cache and skip the task execution.

To enable cache, you must specify the cache storage location by setting IRISML_CACHE_URL environment variable. Currently Azure Blob Storage and local filesystem is supported.

To use Azure Blob Storage, a container URL must be provided. It the URL contains a SAS token, it will be used for authentication. Otherwise, interactive authentication and Managed Identity authentication will be used.

Available official tasks

To show the detailed help for each task, run the following command after installing the package.

irisml_show <task_name>

irisml-tasks

Task	Description
assertion	Test assertion task.
branch	Run different tasks based on a condition.
calculate_cosine_similarity	Calculate a cosine similarity between two tensors.
download_azure_blob	Download a blob from Azure Blob Storage.
get_current_time	Get a current time.
get_dataset_split	Split a dataset into two.
get_dataset_stats	Get statistics of a dataset.
get_dataset_subset	Get a subset of a dataset.
get_fake_image_classification_dataset	Create a fake image classification dataset for testing.
get_fake_object_detection_dataset	Create a fake object detection dataset for testing.
get_item	Get an element from a list.
get_secret_from_azure_keyvault	Get a secret value from Azure KeyVault.
get_topk	Get TopK values from a tensor.
join_filepath	Join path components.
load_state_dict	Load a state_dict into a pytorch model.
print_environment_info	Print information about current environment.
run_parallel	Run multiple tasks in parallel.
run_sequential	Run multiple tasks in sequential.
save_file	Save a binary as a file.
save_state_dict	Save a state_dict of a model as a file.
search_grid_sequential	Grid search hyperparameters.
upload_azure_blob	Upload a blob to Azure Blob Storage.

irisml-tasks-training

This package contains tasks related to pytorch training

Task	Description
append_classifier	Append a classifier layer to a model.
benchmark_model	Get forward/backward pass speed of a model.
build_classification_prompt_dataset	Create a text dataset using a prompt generator.
build_zero_shot_classifier	Generate a zero-shot classification layer.
create_classification_prompt_generator	Create a prompt generator for the classification task.
evaluate_accuracy	Evaluate accuracy of prediction results.
evaluate_detection_average_precision	Evaluate Average Precision of prediction results for the Object Detection task.
export_onnx	Export a pytorch model as ONNX.
get_targets_from_dataset	Extract targets from a dataset.
make_feature_extractor_model	Make a model to extract feature vectors from a model.
make_image_text_contrastive_model	Make a image-text contrastive model.
make_image_text_transform	Make a preprocessing function for image-text dataset.
make_oversampled_dataset	Make a new dataset by oversampling a dataset.
predict	Run inference.
split_image_text_model	Split a image-text model into an image model and a text model.
train	Train a model.
train_with_gradient_cache	Train a model with gradient caching.

irisml-tasks-torchvision

Adapter tasks for torchvision library.

Task	Description
create_torchvision_model	Create a model using the torchvision library.
create_torchvision_transform	Create a preprocessing function using the torchvision library.
load_torchvision_dataset	Load a dataset from the torchvision library.

irisml-tasks-transformers

Adapter tasks for HuggingFace transformers library.

Task	Description
create_transformers_model	Create a model using the transformers library.
create_transformers_tokenizer	Create a tokenizer using the transformers library.

irisml-tasks-timm

Adapter for models in timm library.

Task	Description
create_timm_model	Create a model using the timm library.
create_timm_transform	Create a preprocessing function using the timm library.

irisml-tasks-onnx

Adapter tasks for OnnxRuntime library.

Task	Description
predict_onnx	Run inference for an ONNX model.

irisml-tasks-azureml

Task	Description
run_azureml_child	Run tasks as a new child AzureML Run.
add_aml_tag	Tag the AML Run with a string key and optional value.

irisml-tasks-fiftyone

Task	Description
launch_fiftyone	Launch a fiftyone interface.

Development

Create a new task

To create a Task, you must define a module that contains a "Task" class. Here is a simple example:

# irisml/tasks/my_custom_task.py
import dataclasses
import irisml.core

class Task(irisml.core.TaskBase):  # The class name must be "Task".
  VERSION = '1.0.0'
  CACHE_ENABLED = True  # (default: True) This is optional.

  @dataclasses.dataclass
  class Inputs:  # You can remove this class if the task doesn't require inputs.
    int_value: int
    float_value: float

  @dataclasses.dataclass
  class Config:  # If there is no configuration, you can remove this class. All fields must be JSON-serializable.
    another_float: float
    child_dataclass: dataclass  # If you'd like to define a nested config, you can define another dataclass.

  @dataclasses.dataclass
  class Outputs:  # Can be removed if the task doesn't have outputs.
    float_value: float = 0  # If dry_run() is not implemented, Outputs fields must have default value or default factory.

  def execute(self, inputs: Inputs) -> Outputs:
    return self.Outputs(inputs.int_value * inputs.float_value * self.config.another_float)

  def dry_run(self, inputs: Inputs) -> Outputs:  # This method is optional.
    return self.Outputs(0)  # Must return immediately without actual processing.

Each Task must define "execute" method. The base class has empty implementation for Inputs, Config, Outputs and dry_run(). For the detail, please see the document for TaskBase class.

Related repositories

• irisml-tasks
• irisml-tasks-training
• irisml-tasks-torchvision
• irisml-tasks-transformers
• irisml-tasks-timm
• irisml-tasks-azureml
• irisml-tasks-fiftyone