smartjob 0.0.11

Little async python library for dealing with GCP/Cloud Run Jobs and GCP/VertexAI CustomJobs

Welcome on SmartJob Python Library!

What is it?

SmartJob is a thin Python 3.11+ library for dealing with GCP/Cloud Run Jobs, GCP/VertexAI CustomJobs and (local) Docker:

• in a very simple way
• in a unified way (with as little difference as possible between the 3 providers)
• in a fully async way (in most cases, plain asyncio Python is enough for dealing with complex parallel workflows (including parralelism, chaining, conditionals...) and you don't need to learn another pipeline workflow)
• in a reactive way (when you're in the experimentation phase, you can pass the main Python script without rebuilding/pushing a whole docker image at each attempt)
• in a production ready way (with retries, timeouts...)

Non-features

SmartJob is a thin library and not a whole pipeline framework. He tries to be as un-opinionated as possible.

Quickstart

Let's go 🚀

Documentation

https://fabien-marty.github.io/smartjob/