CLI library for importing, exporting and managing IBM Watson Machine Learning assets.
Project description
AI Lifecycle CLI
CLI allows to export and import assets from Watson Machine Learning service on Cloud Pack for Data and move them between different spaces. It also allows to manage your WML resources such as spaces, assets and deployments. In general, CLI performs import/export jobs in respective APIs and poll the status of the job, until it's completed (regardless of status). We are supporting scenarios, where we can export assets from WML installed on either CPD version 2.5 or 3.0 and import them back to WML installed on CPD version 3.0. During the export, we can choose if either we'd like to export all assets, all assets of given type or just selected assets.
How to install
pip install ai-lifecycle-cli -U
How to run CLI
Example
ai-lifecycle-cli export --url=https://<CP4D_HOST> --user=<CP4D_USER> --pass=<CP4D_PASS> --output_dir=./ --export_json_file=./sources.json
ai-lifecycle-cli import --url=https://<CP4D_HOST> --user=<CP4D_USER> --pass=<CP4D_PASS> --input_file=./archive.zip --import_timeout=1200
Export
You can export assets only from Watson Machine Learning service on CP4D in versions:
- 2.5.0
- 3.0.0
Usage of export
command is shown here:
usage: ai-lifecycle-cli export [-h] --url CPD_URL --user CPD_USER --pass
CPD_PASS --output_dir OUTPUT_DIR
--export_json_file EXPORT_JSON_FILE
[--archive_all] [--export_version {2.5,3.0}]
[--export_timeout EXPORT_TIMEOUT]
[--temp_dir TEMP_DIR]
optional arguments:
-h, --help show this help message and exit
--url CPD_URL URL to CP4D
--user CPD_USER Username used for CP4D login
--pass CPD_PASS Password used for CP4D login
--output_dir OUTPUT_DIR
Directory, where exported content should be saved
--export_json_file EXPORT_JSON_FILE
JSON file, specifying which assets to export from
which projects/spaces
--archive_all Archive all exported content into single ZIP archive
--export_version {2.5,3.0}
Version of exported environment - if not provided,
it'll be auto-detected
--export_timeout EXPORT_TIMEOUT
Set timeout for export job in seconds
--temp_dir TEMP_DIR Directory for temporary files
List of supported assets
List with supported assets for export with their respective names in API:
- Models:
wml_model
- Python Functions:
wml_function
- Pipelines:
wml_pipeline
- Model Definitions:
wml_model_definitions
- Experiments:
wml_experiment
- Python Scripts:
script
- Shiny Apps:
shiny_asset
How to specify, which assets do you want to export
Flag --export_json_file EXPORT_JSON_FILE
is responsible for passing the file with specification of which aasets should be exported from which spaces in source environment.
Syntax of such file looks like this:
{
"space": [
{
"guid": "<space_guid_1>"
},
...
{
"guid": "<space_guid_2>",
"assets": {
"<asset_type_1>": "all",
"<asset_type_2>": ["asset_guid_1", ...]
}
},
...
]
}
You need to provivde JSON file with key named space
- under this key you specify array of space items, which can be defined in few ways:
- To specify that you want to export all assets from given space (i.e.
<space_guid_1>
), space item in array would look like this:
{
"guid": "<space_guid_1>"
}
- To specify that you want to export all assets of given type (i.e.
<asset_type_1>
from given space (i.e.<space_guid_2>
), space item in array would look like this:
{
"guid": "<space_guid_2>",
"assets": {
"<asset_type_1>": "all"
}
}
- To specify that you want to export specified assets (i.e.
"asset_guid_1"
and"asset_guid_2"
of given type (i.e.<asset_type_2>
from given space (i.e.<space_guid_3>
), space item in array would look like this:
{
"guid": "<space_guid_3>",
"assets": {
"<asset_type_2>": ["asset_guid_1", "asset_guid_2"]
}
}
Sample files to use for --export_json_file
are under samples
directory.
Import
You can import assets only to Watson Machine Learning service on CP4D in versions:
- 3.0.0
Usage of import
command is shown here:
usage: ai-lifecycle-cli import [-h] --url CPD_URL --user CPD_USER --pass
CPD_PASS --input_file INPUT_FILE
[--import_version {2.5,3.0}]
[--import_name IMPORT_NAME]
[--import_desc IMPORT_DESC]
[--import_timeout IMPORT_TIMEOUT]
[--temp_dir TEMP_DIR]
optional arguments:
-h, --help show this help message and exit
--url CPD_URL URL to CP4D (default: None)
--user CPD_USER Username used for CP4D login (default: None)
--pass CPD_PASS Password used for CP4D login (default: None)
--input_file INPUT_FILE
Path to exported archive file to be imported (default:
None)
--import_version {2.5,3.0}
Version of imported environment - if not provided,
it'll be auto-detected (default: 3.0)
--import_name IMPORT_NAME
Name of created container (e.g. space) during import
(default: None)
--import_desc IMPORT_DESC
Description of created container (e.g. space) during
import (default: None)
--import_timeout IMPORT_TIMEOUT
Set timeout for export job in seconds (default: 600)
--temp_dir TEMP_DIR Directory for temporary files (default: /var/tmp/ai-
lifecycle-cli)
Managing WML resources
AI Lifecycle CLI also supports simple management for WML assets and deployments.
In sections below, there's a list of supported commands for WML asset and deployment management.
To get more information on the commands, use -h/--help
flag with the command.
Assets
You can perform the following operations:
asset list
- list all assets persisted in the deployment space
Deployments
You can perform the following operations:
deployment list
- list all deployments persisted in the deployment spacedeployment create
- create new deployment in the deployment spacedeployment delete
- delete deployment from the deployment space
Spaces
You can perform the following operations:
space list
- list all deployment spacesspace delete
- delete deployment space
Deployment jobs
You can perform the following operations:
deployment-job list
- list all deployment jobs in the deployment spacedeployment-job create
- create new deployment job in the deployment spacedeployment-job cancel
- cancel running deployment job in the deployment space
Notes & Limitations
Supporting for now only:
- export from CP4D version 2.5 and 3.0
- import to CP4D version 3.0
Flag --export-timeout
is not applicable for exports from CP4D version 2.5.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file ai-lifecycle-cli-1.0.0.tar.gz
.
File metadata
- Download URL: ai-lifecycle-cli-1.0.0.tar.gz
- Upload date:
- Size: 27.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.4.0.post20200518 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.7.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | bf3755f8537258859021fdf9a3cfa22be52533043aa49f15b00f41226cdd6700 |
|
MD5 | 0757858a835bf06ed53f27f2b364b39d |
|
BLAKE2b-256 | c7dfd306d525e62abe3d5f0c8e29ecfb031ed51653a390f0210e0d2d5bc71c90 |
File details
Details for the file ai_lifecycle_cli-1.0.0-py3-none-any.whl
.
File metadata
- Download URL: ai_lifecycle_cli-1.0.0-py3-none-any.whl
- Upload date:
- Size: 41.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.4.0.post20200518 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.7.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97b380a222134da0f958b5d9b16076a57b9354363466aa177561027c3d1d9687 |
|
MD5 | caa0e78c2518a9b3cab3df9216f8a313 |
|
BLAKE2b-256 | be7f4629cf4327eaec20d93e92ac4394946c04c1e086dc4c5847be0b415f976a |