Guillotina addon to split actions into smaller bits
Project description
Introduction
============
`guillotina_hive` is a task runner whose main goal is to provide a mechanism
to iterate through all the content on a database as quickly as possible.
To accomplish that end, hive integrates with container orchestrators(k8s or nomad)
to schedule jobs on and retrieve results from those jobs.
Installation
------------
With pip:
pip install guillotina_hive
Guillotina configuration
------------------------
Example here with json::
"applications": ["guillotina_hive"],
"hive": {
"default_image": None,
"default_namespace": "hive",
"orchestrator": "k8s",
"cluster_config": {},
"guillotina_default": {
"entrypoint": None,
"volumes": None,
"volumeMounts": None,
"envFrom": None
},
'quota': None
}
Settings explained
~~~~~~~~~~~~~~~~~~
- default_image: default image to use for jobs to run with
- default_namespace: cluster namespace to use
- orchestrator: k8s or nomad
- cluster_config: what to configure aioclustermanager with
- guillotina_default: default configuration for guillotina jobs
- quota: define quote on cluster namespace
Defining a job
--------------
We use decorators to provide tasks::
from guillotina_hive.decorators import hive_task
@hive_task(name='something')
async def something(arg1, arg2):
return foobar
You can also use application settings::
{
"calculate-numbers": "guillotina_hive.tests.tasks.calculate_numbers"
}
1.0.12 (2018-05-05)
-------------------
- job name should not be the id we give it
[vangheem]
- Be able to customize default container args
[vangheem]
1.0.11 (2018-04-27)
-------------------
- Updating aioclustermanager dependency
[ramon]
1.0.10 (2018-04-26)
-------------------
- Remove print
[vangheem]
1.0.9 (2018-04-26)
------------------
- apply builtin tasks setup request object url as well
[vangheem]
1.0.7 (2018-04-25)
------------------
- improve user login for apply tasks
[vangheem]
1.0.4 (2018-04-24)
------------------
- Fix working with decorators
[vangheem]
1.0.3 (2018-04-24)
=======
- fix to work with guillotina 3
[vangheem]
1.0.2 (27/04/2018)
------------------
- Updating aioclustermanager dependency
[ramon]
1.0.1 (9/4/2018)
----------------
- Adding manifest
[ramon]
1.0.0 (5/4/2018)
----------------
- Initial version
============
`guillotina_hive` is a task runner whose main goal is to provide a mechanism
to iterate through all the content on a database as quickly as possible.
To accomplish that end, hive integrates with container orchestrators(k8s or nomad)
to schedule jobs on and retrieve results from those jobs.
Installation
------------
With pip:
pip install guillotina_hive
Guillotina configuration
------------------------
Example here with json::
"applications": ["guillotina_hive"],
"hive": {
"default_image": None,
"default_namespace": "hive",
"orchestrator": "k8s",
"cluster_config": {},
"guillotina_default": {
"entrypoint": None,
"volumes": None,
"volumeMounts": None,
"envFrom": None
},
'quota': None
}
Settings explained
~~~~~~~~~~~~~~~~~~
- default_image: default image to use for jobs to run with
- default_namespace: cluster namespace to use
- orchestrator: k8s or nomad
- cluster_config: what to configure aioclustermanager with
- guillotina_default: default configuration for guillotina jobs
- quota: define quote on cluster namespace
Defining a job
--------------
We use decorators to provide tasks::
from guillotina_hive.decorators import hive_task
@hive_task(name='something')
async def something(arg1, arg2):
return foobar
You can also use application settings::
{
"calculate-numbers": "guillotina_hive.tests.tasks.calculate_numbers"
}
1.0.12 (2018-05-05)
-------------------
- job name should not be the id we give it
[vangheem]
- Be able to customize default container args
[vangheem]
1.0.11 (2018-04-27)
-------------------
- Updating aioclustermanager dependency
[ramon]
1.0.10 (2018-04-26)
-------------------
- Remove print
[vangheem]
1.0.9 (2018-04-26)
------------------
- apply builtin tasks setup request object url as well
[vangheem]
1.0.7 (2018-04-25)
------------------
- improve user login for apply tasks
[vangheem]
1.0.4 (2018-04-24)
------------------
- Fix working with decorators
[vangheem]
1.0.3 (2018-04-24)
=======
- fix to work with guillotina 3
[vangheem]
1.0.2 (27/04/2018)
------------------
- Updating aioclustermanager dependency
[ramon]
1.0.1 (9/4/2018)
----------------
- Adding manifest
[ramon]
1.0.0 (5/4/2018)
----------------
- Initial version
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
guillotina_hive-1.0.12.tar.gz
(16.3 kB
view details)
Built Distribution
File details
Details for the file guillotina_hive-1.0.12.tar.gz
.
File metadata
- Download URL: guillotina_hive-1.0.12.tar.gz
- Upload date:
- Size: 16.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | da32a58a2ec234f8c065c435bf94ac01ae0ceb8d6fece020dc4f34a914fd309e |
|
MD5 | fbe59453deec4608c1e3b7811875a748 |
|
BLAKE2b-256 | d2670e9ce9af2052a5f20aa22b33fcf5529364fb1f8ecc23dc20a908fad45fde |
File details
Details for the file guillotina_hive-1.0.12-py3-none-any.whl
.
File metadata
- Download URL: guillotina_hive-1.0.12-py3-none-any.whl
- Upload date:
- Size: 25.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9db3b04bfbb6ad5a321b58684cbd0dfc4ebf5601402d52eee648cb8f3adbd8c |
|
MD5 | 9dba099af53aab908a7fb7457b4e12e3 |
|
BLAKE2b-256 | dfe82a0e8f5c855495dc4a60a156aeb1b3c3182ebdd2e61a844605b351e05155 |