arm-avhclient 0.1.10

Arm Virtual Hardware Command Line Interface

Arm Virtual Hardware Client (avhclient)

The Arm Virtual Hardware Client (avhclient) is a python module that provides an interface for deploying and using Arm Virtual Hardware (AVH).

It enables uniform implementation of CI operations in various environments with reference examples provided for the following use cases:

• Jenkins CI pipelines

• GitHub-Actions workflows

• Local use with AVH targets

Other environments can be supported using demonstrated concepts as well.

Example projects using AVH Client

• AVH-GetStarted

• CMSIS-RTOS2-Validation

• TensorFlow Lite for Microcontrollers

---

Installation

Local installation

Installing (development snapshot) directly from GitHub main branch:

pip install git+https://github.com/ARM-software/avhclient.git@main

Docker container

Instead of installing Python and the AVH Client module into the local environment one can use pre-built Docker images:

docker pull ghcr.io/arm-software/avhclient

---

Backend Setup

avhclient can control different backends with Arm Virtual Hardware. Following options are currently available:

• aws (default) - interacts with AVH AMI available through AWS Marketplace

• local - operates with AVH Targets installed locally.

The backend can be specified with -b option preceding the actual avhclient command.

Depending on the backend certain environment setup is expected.

AWS Backend Setup

avhclient accesses AWS services via Boto3 AWS SDK and for that requires a set of parameters to be available in the environment.

AWS Credentials

It is necessary to expose avhclient with the AWS credentials for your account. You can either export your AWS credentials:

export AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID"
export AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_ACCESS_KEY"
export AWS_SESSION_TOKEN="YOUR_AWS_SESSION_TOKEN"

or create a AWS credential file on ~/.aws/credentials (Linux & Mac) or %USERPROFILE%.awscredentials (Windows):

[default]
aws_access_key_id=YOUR_AWS_ACCESS_KEY_ID
aws_secret_access_key=YOUR_AWS_SECRET_ACCESS_KEY
aws_session_token=YOUR_AWS_SESSION_TOKEN

More info AWS CLI config and credentials

AWS Account info

In order for avhclient to create or access an AVH instance following parameters need to be defined in the execution environment of avhclient:

• When creating and running a new AVH instance:

  Mandatory info:

  export AWS_IAM_PROFILE='YOUR_IAM_PROFILE'
  export AWS_SECURITY_GROUP_ID='YOUR_AWS_SECURITY_GROUP_ID'
  export AWS_SUBNET_ID='YOUR_SECURITY_GROUP_ID'
  export AWS_S3_BUCKET_NAME='YOUR_B3_BUCKET_NAME'

  Optional info (examples):

  export AWS_AMI_ID=DESIRED_AVH_AMI_ID
  export AWS_AMI_VERSION=1.1.2
  export AWS_EFS_DNS_NAME=fs-066cf410af2428e2f.efs.eu-west-1.amazonaws.com
  export AWS_EFS_PACKS_DIR=packs
  export AWS_KEEP_EC2_INSTANCES=true
  export AWS_KEY_NAME=YOUR_AWS_KEYPAIR_NAME
  export AWS_INSTANCE_TYPE=t2.micro
  export AWS_INSTANCE_NAME=MY_AVH_INSTANCE

  • If AWS_AMI_VERSION is not set, the avhclient will use the latest available version of AVH AMI.

  • If AWS_EFS_DNS_NAME is set, the AVH Client will try to mount it during the cloud-init phase. The only scenario supported for now is using Packs.

  • If AWS_EFS_PACKS_DIR is set, the mount path is relative to /home/ubuntu folder. Default folder is packs and if it exists locally will be then replaced by the EFS mount. Only used when AWS_EFS_DNS_NAME env is set.

  AWS Cloudformation can be used to create the AWS resources required for AVH operation, as shown in this template

• When reusing an AVH Instance:

  export AWS_INSTANCE_ID=YOUR_INSTANCE

Local Backend Setup

Operation with a local backend requires no specific environment parameters, but assumes that necessary toolchain, AVH targets and utilities are installed locally on the machine and configured for execution in command line.

---

Usage

Getting Help

To get the brief descriptions of all commands and options available with avhclient execute:

avhclient -h

You can also use option -h with a specific command to get help for it. For example for execute command:

avhclient execute -h

Execute command

• Create a new AWS AVH instance and run AVH project

  The execute command bundles all necessary steps to build your avh project:

  • prepare the backend.

  • upload your files

  • run your commands

  • download the results

  • cleanup the backend

  Inform the path for the avh.yml file for your AVH project (example):

  avhclient -b aws execute --specfile AVH-GetStarted/basic/avh.yml

• You can also run in AVH commands in your local computer by selecting local backend:

  avhclient -b local execute --specfile AVH-GetStarted/basic/avh.yml

• There are also backend specific info you can provide to the tool:

  avhclient -b aws -h (get full list)
  avhclient -b aws --instance-name MY_NEW_NAME execute --specfile AVH-GetStarted/basic/avh.yml (seeting a new AVH instance name)
  avhclient -b aws --ami-version 1.1.0 --specfile AVH-GetStarted/basic/avh.yml (Create a new AVH instance from a v1.1.0 AVH AMI)
  avhclient -b aws --ami-version >1.1.0 --specfile AVH-GetStarted/basic/avh.yml (Create a new AVH instance from a >v1.1.0 AVH AMI)

Execute with Docker

To run avhclient in a Docker container one needs to create an environment file (env.txt) with the following content:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_IAM_PROFILE
AWS_SECURITY_GROUP_ID
AWS_SUBNET_ID
AWS_S3_BUCKET_NAME
AWS_DEFAULT_REGION
AWS_AMI_ID
AWS_AMI_VERSION
AWS_KEEP_EC2_INSTANCES
AWS_KEY_NAME
AWS_INSTANCE_TYPE
AWS_INSTANCE_NAME

This environment file is used to forward the local environment variables into the Docker container. Having this prepared one can run avhclient in a container as follows:

docker run --rm -i --env-file ./env.txt \
    -v $(pwd):/workspace \
    -w /workspace \
    ghcr.io/arm-software/avhclient \
    avhclient [..]

The arguments are the same as above. If one requires more files from the Docker host to be mapped into the container, this can be done like:

docker run --rm -i --env-file ./env.txt \
    -v $HOME/.ssh:/root/.ssh \
    -v $HOME/.aws:/root/.aws \
    -v $(pwd):/workspace \
    -w /workspace \
    ghcr.io/arm-software/avhclient \
    avhclient [..]

This exposes the local user’s SSH and AWS config files to the container.

---

AVH YML file syntax

avhclient execute command requires a specfile in YML format that describes details of individual steps to be executed on AVH. The file syntax is explained below.

A JSON schema for automatic checks and auto-completion is in schema/avh.schema.json.

Fields

Format of the specfile:
        name: (optional) The name of the workload.
        workdir: (optional) The local directory to use as the workspace, defaults to specfile's parent.
        backend: (optional) Dictionary with backend specific parameters.
          aws: (optional) Dictionary with AWS backend specific parameters. (see backend help)
          local: (optional) Dictionary with local backend specific parameters. (see backend help)
        upload: (optional) List of glob patterns of files (relative to workdir) to be sent to the AVH backend. (see glob format)
        steps: (mandatory) List of steps to be executed on the AVH backend.
          - run: String written into a bash script and executed on the AVH backend inside the workspace directory.
        download: (optional) List of glob patterns of files (relative to workdir) to be retrieved back from the AVH backend. (see glob format)
    Glob format:
        The list of glob patterns is evaluated in order.
        Wildcard '*' matches all files but no directory except hidden files (starting with '.').
        Wildcard '**' matches all files and directories except hidden files/directories (starting with '.').
        Inclusive matches (no prefix) are added to the file list.
        Exclusive (prefixed with '-:') matches are removed from current file list.

Example

# yaml-language-server: $schema=https://raw.githubusercontent.com/ARM-software/avhclient/main/schema/avh.schema.json

name: "AVH GetStarted Example"
workdir: ./
backend:
  aws:
    ami-version: ~=1.1
    instance-type: t2.micro
upload:
  - RTE/**/*
  - -:RTE/**/RTE_Components.h
  - basic.debug.cprj
  - build.py
  - main.c
  - requirements.txt
  - retarget_stdio.c
  - vht_config.txt
  - README.md
steps:
  - run: |
      pip install -r requirements.txt
      python build.py --verbose build run
download:
  - RTE/**/RTE_Components.h
  - Objects/basic.axf
  - Objects/basic.axf.map
  - basic-*.xunit
  - basic-*.zip