Hoodat Pipeline Components

This repository provides an SDK and a set of components that perform tasks in hoodat.

It is modelled after this repository of shared components for GCP: https://github.com/kubeflow/pipelines/tree/google-cloud-pipeline-components-1.0.1/components/google-cloud

To create new release of package and push to pypi:

Update the version with commitizen:

# cz bump --dry-run
cz bump

Push to main branch

git push

Create a new release in github.

The package will be built and pushed to pypi in a github action.

## Makefile

There is a Makefile at the root of this project which provides some useful functionality for developing and publishing components. In the next sections of this document some of this funcitonality will be described.

Important to the use of the Makefile is the creation of an env.sh file with necessary arguments populated. See env.sh.example for an example of what this file should look like. Copy it to env.sh and replace the default arguments with your own.

To create a new component with your own Dockerfile

New components should be added to the hoodat_vertex_components/components subdirectory. See already existing examples. Here is a common file structure for a component:

├── make_cascade_file
│   ├── Dockerfile
│   ├── cascades.csv
│   ├── component.yaml
│   ├── make_cascade_file.py
│   ├── poetry.lock
│   ├── pyproject.toml
│   └── tests
│       └── test_filter_cascades.py

To run a components docker container in interactive mode

This function will be useful for running a components docker image interactively. Update the env.sh with the name of the component and run:

make run_interactive

To run a pipeline with a single component in it

It may be useful to test a component in a pipeline. To do this, update the env.sh with the name of the component and run:

make push_and_pipeline

To create a new python component

Look at video_to_frames for an example.

Once you're happy, run:

COMPONENT_NAME=video_to_frames
cd hoodat_vertex_components/components/$COMPONENT_NAME
poetry run python component.py

Release files for hoodat-vertex-components 1.6.6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for hoodat-vertex-components 1.6.6
File	Size	Uploaded
hoodat_vertex_components-1.6.6.tar.gz	3.4 MB	Apr 23, 2023	Details

Built distribution (wheel)

Table of built distributions (wheels) for hoodat-vertex-components 1.6.6
File	Interpreter	ABI	Platform	Reset
hoodat_vertex_components-1.6.6-py3-none-any.whl 3.4 MB Apr 23, 2023	Python 3	none	any	Details

Total release size: 6.7 MB

Release files / hoodat_vertex_components-1.6.6.tar.gz

Download URL	hoodat_vertex_components-1.6.6.tar.gz
Size	3.4 MB
Tags	Source
SHA-256 checksum How to use checksums	`c8a6babce62c57072b826320ce9a06d1020b21b39bbce9c4fd1ac50da6539a73`
BLAKE2b-256 checksum How to use checksums	`71fc278b099231e7a4bbbb10fc9bbd26ec4f7869832989db6b274e47985eec78`
Upload date	Apr 23, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.4.2 CPython/3.8.16 Linux/5.15.0-1035-azure`

Release files / hoodat_vertex_components-1.6.6-py3-none-any.whl

Download URL	hoodat_vertex_components-1.6.6-py3-none-any.whl
Size	3.4 MB
Tags	Python 3
SHA-256 checksum How to use checksums	`332b6e9e12d8ec46b710e3e1d020463e559b86926ed78b6317630b69d2d5f471`
BLAKE2b-256 checksum How to use checksums	`6f0ff7eb9aad677583bde4945ed8593e75eb438e5385e18f0fc8b4b0769976d0`
Upload date	Apr 23, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.4.2 CPython/3.8.16 Linux/5.15.0-1035-azure`