VDK Control CLI allows user to create, delete, manage and their Data Jobs in Kubernetes runtime.
Project description
Versatile Data Kit Control CLI
VDK Control CLI is meant for Data Engineers to use to manage the lifecycle of jobs - create, delete, deploy, configure Data Jobs.
To build or contribute, see CONTRIBUTING.md.
Installation
Install VDK Control CLI with:
pip install vdk-control-cli
This will install console application called vdkcli
Then run help to see what it can do:
vdkcli --help
vdkcli
is the name of the console application only when vdk-control-cli is installed autonomously. Typically,
it is a dependency of Versatile Data Kit and the console application is vdk
(hence, all commands in error and help
messages in this project refer to vdk
). Keep in mind that if you are using this project autonomously, you should
use vdkcli
command instead of vdk
.
Environment variables:
-
VDK_BASE_CONFIG_FOLDER - Override local base configuration folder (by default in $HOME folder). Inside it will create folder .vdk.internal. CLI state may be kept there (login info). Use in case multiple users need to login (e.g in case of automation) on same machine.
-
VDK_CONTROL_SERVICE_REST_API_URL - Default Control Service URL to use if not specified as command line argument
-
VDK_API_TOKEN - Default API Token to use if another authentication has not been used with vdk login
-
VDK_API_TOKEN_AUTHORIZATION_URL - Default API token URL to use if another authentication has not been used with vdk login.
Security
If Control Service configured require authentication: vdk login must have finished successfully.
Or alternatively correct VDK_API_TOKEN_AUTHORIZATION_URL and VDK_API_TOKEN must be set correctly and will behave same as vdk login -t api-token
.
If vdk login is used - it take priority over environment variables set VDK_API_TOKEN_AUTHORIZATION_URL and VDK_API_TOKEN
To clear previous login info (aka logout) use vdk logout
.
In case of credentials type vdk login flow we start a process on port 31113
to receive the credentials.
If you already have process running on 31113
you can override the value.
To override the port set environmental variable OAUTH_PORT
with free port which the client can use.
Plugins
Installing and Using plugins
Installing a third party plugin can be easily done with pip:
pip install vdk-control-cli-NAME
pip uninstall vdk-control-cli-NAME
If a plugin is installed, vdk automatically finds and integrates it.
Write your own plugin
A plugin is python module that enhances or changes the behaviour of the vdk cli.
A plugin contains one or multiple hook functions.
See all supported hook function specifications that can be implemented in specs.py
In order to create a new plugin there are only 2 steps:
- Create your implementation of the plugin's hook(s):
# define hookimpl as follows (library requirement: pluggy)
hookimpl = pluggy.HookimplMarker("vdk_control_cli.plugin")
# though it's better to use `vdk.internal.control.plugin.markers.hookimpl` from vdk-control-cli python package
# name of function must match name of hookspec function
@hookimpl
def get_default_commands_options():
# your implementation here ; for example to set defaults for `vdk login --type --oauth2-authorization-url` command
default_options = {
"login": {
"auth_type": "api-token", # note values must be valid or the plugin may break the CLI, no checking is done at this point
"api_token_authorization_url": "http://localhost/authorize" # replace dashes with underscore for the argument name
}
}
return default_options
- Register as plugin by listing the plugin modules in vdk_control_cli.plugin entry_point in your setup.py:
entry_points={ 'vdk_control_cli.plugin': ['name_of_plugin = myproject.pluginmodule'] }
The plugin system is based on pluggy.
SDK Extensibility design can be seen here
Authentication
In order to use credentials login type you need to create OAuth2 Application.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for vdk_control_cli-1.3.1406932984.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e825ea0747a2ff96585c8afbeb6c839dd7cdc912a208b7ea2a867f7ee0625a45 |
|
MD5 | 3c3e8711dde2818488aac640308b6105 |
|
BLAKE2b-256 | e7881e3c323f72510857c913e63508877ad8b13c0321ac08e7ddfd24256e2bb8 |