Skip to main content

Guillotina addon to split actions into smaller bits

Project description


`guillotina_hive` is a task runner whose main goal is to provide a mechanism
to iterate through all the content on a database as quickly as possible.

To accomplish that end, hive integrates with container orchestrators(k8s or nomad)
to schedule jobs on and retrieve results from those jobs.


With pip:

pip install guillotina_hive

Guillotina configuration

Example here with json::

"applications": ["guillotina_hive"],
"hive": {
"default_image": None,
"default_namespace": "hive",
"orchestrator": "k8s",
"cluster_config": {},
"guillotina_default": {
"entrypoint": None,
"volumes": None,
"volumeMounts": None,
"envFrom": None
'quota': None

Settings explained

- default_image: default image to use for jobs to run with
- default_namespace: cluster namespace to use
- orchestrator: k8s or nomad
- cluster_config: what to configure aioclustermanager with
- guillotina_default: default configuration for guillotina jobs
- quota: define quote on cluster namespace

Defining a job

We use decorators to provide tasks::

from guillotina_hive.decorators import hive_task
async def something(arg1, arg2):
return foobar

You can also use application settings::

"calculate-numbers": "guillotina_hive.tests.tasks.calculate_numbers"

1.0.15 (2018-05-19)

- Downgrade guillotina test requirements

1.0.14 (2018-05-19)

- upgrade guillotina in reqs

1.0.13 (2018-05-19)

- Do not log cancelled errors on init

1.0.12 (2018-05-05)

- job name should not be the id we give it

- Be able to customize default container args

1.0.11 (2018-04-27)

- Updating aioclustermanager dependency

1.0.10 (2018-04-26)

- Remove print

1.0.9 (2018-04-26)

- apply builtin tasks setup request object url as well

1.0.7 (2018-04-25)

- improve user login for apply tasks

1.0.4 (2018-04-24)

- Fix working with decorators

1.0.3 (2018-04-24)

- fix to work with guillotina 3

1.0.2 (27/04/2018)

- Updating aioclustermanager dependency

1.0.1 (9/4/2018)

- Adding manifest

1.0.0 (5/4/2018)

- Initial version

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
guillotina_hive-1.0.15-py3-none-any.whl (25.7 kB) Copy SHA256 hash SHA256 Wheel py3
guillotina_hive-1.0.15.tar.gz (16.4 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page