Flyte agent for Perian Job Platform (perian.io)
Project description
Flytekit Perian Job Platform Plugin
Flyte Agent plugin for executing Flyte tasks on Perian Job Platform (perian.io).
Perian Job Platform is still in closed beta. Contact support@perian.io if you are interested in trying it out.
To install the plugin, run the following command:
pip install flytekitplugins-perian-job
Getting Started
This plugin allows executing PythonFunctionTask
on Perian.
An ImageSpec need to be built with the perian agent plugin installed.
Parameters
The following parameters can be used to set the requirements for the Perian task. If any of the requirements are skipped, it is replaced with the cheapest option. At least one requirement value should be set.
cores
: Number of CPU coresmemory
: Amount of memory in GBaccelerators
: Number of acceleratorsaccelerator_type
: Type of accelerator (e.g. 'A100'). For a full list of supported accelerators, use the perian CLI list-accelerators command.country_code
: Country code to run the job in (e.g. 'DE')
Credentials
The following secrets are required to be defined for the agent server:
- Perian credentials:
perian_organization
perian_token
- AWS credentials for accessing the Flyte storage bucket. You might need to create long-lived credentials to avoid expiry when running tasks. These credentials are never logged by Perian and are stored only until it is used, then immediately deleted:
aws_access_key_id
aws_secret_access_key
- (Optional) Custom docker registry for pulling the Flyte image:
docker_registry_url
docker_registry_username
docker_registry_password
Example
example.py
workflow example:
from flytekit import ImageSpec, task, workflow
from flytekitplugins.perian_job import PerianConfig
image_spec = ImageSpec(
name="flyte-test",
registry="my-registry",
python_version="3.11",
apt_packages=["wget", "curl", "git"],
packages=[
"flytekitplugins-perian-job",
],
)
@task(container_image=image_spec,
task_config=PerianConfig(
accelerators=1,
accelerator_type="A100",
))
def perian_hello(name: str) -> str:
return f"hello {name}!"
@workflow
def my_wf(name: str = "world") -> str:
return perian_hello(name=name)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-perian_job-1.12.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c142eecb85b01951fd299f5fc5b6f9cadecb9e50db0adc43c3b62d1f6ef7176e |
|
MD5 | 28f7a1be8d62321f6dcb1f3eec0ba58b |
|
BLAKE2b-256 | e690e82d4f9a910b92bf55f421d630a9f0bdecfdd80a82a5b014dfaebe57d630 |
Hashes for flytekitplugins_perian_job-1.12.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3ec9feeb650c4cb8eefc917cbc0207f0e282755561e87159e544f8e5c83c7b24 |
|
MD5 | ad8b8f0cc4969373ca7a747afc5f64a7 |
|
BLAKE2b-256 | 5b20e91c371d08ae5cb4ab0844928ff94158c3a9e8ff1113afb36f8f44bea73b |