This package allows a user to build custom GBDX tasks that can be executed locally and also remotely on the platform. All from a python file and commandline tools.
To run remote tasks, this package uses the gbdx-auth package. Therefore the users credentials must be configured according to gbdx-auth.
Installation
To install from pypi:
pip install gbdx-cloud-harness
Usage
The cloud-harness package has a template task with example usage inside. To create a new template:
cloud-harness create my_app cd my_app/
In the new folder, there will be a app.py file which is the template. Open in your favorite editor to start building a new task. When ready to run the task:
cloud-harness run app.py
Please note, to run the task locally, all port values must be valid filesystem locations. Otherwise errors will be raised.
When the task is ready to be ran on the platform, use the –remote flag:
cloud-harness run app.py --remote
This will run the task on the platform assuming all the port values are S3 locations. Otherwise errors will be raised.
If there is local data that needs to be pushed to S3 for the remote run, then the –upload flag needs to be used:
cloud-harness run app.py --remote --upload
This will push all local data the ports contain to the users account storage, prior to executing the workflow.
Development
Contributing
Please contribute! Please make pull requests directly to master. Before making a pull request, please:
Ensure that all new functionality is covered by unit tests.
Verify that all unit tests are passing.
Ensure that all functionality is properly documented.t
Ensure that all functions/classes have proper docstrings so sphinx can autogenerate documentation.
Fix all versions in setup.py (and requirements.txt)
Run Tests
Tests use pytest framework
py.test [...] python -m pytest [...]
Create a new version
To create a new version:
bumpversion ( major | minor | patch ) git push --tags
Don’t forget to update the changelog and upload to pypi.
1 maintainer
Metadata
1 maintainer
Release files for gbdx-cloud-harness 0.2.25
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| gbdx-cloud-harness-0.2.25.tar.gz | 51.0 kB | Details |
Release files / gbdx-cloud-harness-0.2.25.tar.gz
| Download URL | gbdx-cloud-harness-0.2.25.tar.gz |
|---|---|
| Size | 51.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d947da3e353cf846564d673ae04b561e2468299a344c4f747d76e32a464059ac
|
|
BLAKE2b-256 checksum How to use checksums |
9ab36f5c12bb5d97bd8a63a7719cba61e525f5d59a276fd7822348d36e3d883a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
