Skip to main content

Guillotina addon to split actions into smaller bits

Project description

Introduction
============

`guillotina_hive` is a task runner whose main goal is to provide a mechanism
to iterate through all the content on a database as quickly as possible.

To accomplish that end, hive integrates with container orchestrators(k8s or nomad)
to schedule jobs on and retrieve results from those jobs.

Installation
------------

With pip:

pip install guillotina_hive


Guillotina configuration
------------------------

Example here with json::

"applications": ["guillotina_hive"],
"hive": {
"default_image": None,
"default_namespace": "hive",
"orchestrator": "k8s",
"cluster_config": {},
"guillotina_default": {
"entrypoint": None,
"volumes": None,
"volumeMounts": None,
"envFrom": None
},
'quota': None
}


Settings explained
~~~~~~~~~~~~~~~~~~

- default_image: default image to use for jobs to run with
- default_namespace: cluster namespace to use
- orchestrator: k8s or nomad
- cluster_config: what to configure aioclustermanager with
- guillotina_default: default configuration for guillotina jobs
- quota: define quote on cluster namespace


Defining a job
--------------

We use decorators to provide tasks::

from guillotina_hive.decorators import hive_task
@hive_task(name='something')
async def something(arg1, arg2):
return foobar


You can also use application settings::

{
"calculate-numbers": "guillotina_hive.tests.tasks.calculate_numbers"
}

1.0.15 (2018-05-19)
-------------------

- Downgrade guillotina test requirements


1.0.14 (2018-05-19)
-------------------

- upgrade guillotina in reqs


1.0.13 (2018-05-19)
-------------------

- Do not log cancelled errors on init
[vangheem]


1.0.12 (2018-05-05)
-------------------

- job name should not be the id we give it
[vangheem]

- Be able to customize default container args
[vangheem]

1.0.11 (2018-04-27)
-------------------

- Updating aioclustermanager dependency
[ramon]

1.0.10 (2018-04-26)
-------------------

- Remove print
[vangheem]


1.0.9 (2018-04-26)
------------------

- apply builtin tasks setup request object url as well
[vangheem]


1.0.7 (2018-04-25)
------------------

- improve user login for apply tasks
[vangheem]


1.0.4 (2018-04-24)
------------------

- Fix working with decorators
[vangheem]


1.0.3 (2018-04-24)
=======

- fix to work with guillotina 3
[vangheem]

1.0.2 (27/04/2018)
------------------

- Updating aioclustermanager dependency
[ramon]


1.0.1 (9/4/2018)
----------------

- Adding manifest
[ramon]

1.0.0 (5/4/2018)
----------------

- Initial version


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

guillotina_hive-1.0.15.tar.gz (16.4 kB view details)

Uploaded Source

Built Distribution

guillotina_hive-1.0.15-py3-none-any.whl (25.7 kB view details)

Uploaded Python 3

File details

Details for the file guillotina_hive-1.0.15.tar.gz.

File metadata

File hashes

Hashes for guillotina_hive-1.0.15.tar.gz
Algorithm Hash digest
SHA256 1c92996cf0f48e7ef6dc8fa4fde9c93a78f50c4eaed4945ab829b2ca29b3b986
MD5 c4b34b2e304f96ce2dea2b670d8525f7
BLAKE2b-256 caaeefe3ecda619d4cfc31084d2e5a118c4b4200caeeb8e4cef10713e0b47394

See more details on using hashes here.

File details

Details for the file guillotina_hive-1.0.15-py3-none-any.whl.

File metadata

File hashes

Hashes for guillotina_hive-1.0.15-py3-none-any.whl
Algorithm Hash digest
SHA256 bf34da63a23536339c4849e214b5958900723e2d80650fe1ea8e4b287c3ba9be
MD5 284b5bb1e10d9f659ad4f4786462ec66
BLAKE2b-256 ff8e7668c1f08e1310459a2cfa0056202805552651dee91bd49413bf16b303ee

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page