A package providing template applications for data services, and a python SDK to the Octue API
Project description
octue-sdk-python
Utilities for running python based data services, digital twins and applications with the Octue toolkit and twined SDK for python based apps running within octue.
Installation and usage
For usage as a scientist or engineer, run the following command in your environment:
pip install octue
The command line interface (CLI) can then be accessed via:
octue-app --help
Developer notes
Installation
For development, run the following from the repository root, which will editably install the package:
pip install -r requirements-dev.txt
Testing
These environment variables need to be set to run the tests:
GOOGLE_APPLICATION_CREDENTIALS=/absolute/path/to/service/account/file.json
TEST_PROJECT_NAME=<name-of-google-cloud-project-to-run-pub-sub-tests-on>
Then, from the repository root, run
python3 -m unittest
Documentation for use of the library is here. You don't need to pay attention to the following unless you plan to develop octue-sdk-python
itself.
Pre-Commit
You need to install pre-commit to get the hooks working. Do:
pip install pre-commit
pre-commit install
Once that's done, each time you make a commit, the following checks are made:
- valid github repo and files
- code style
- import order
- PEP8 compliance
- documentation build
- branch naming convention
Upon failure, the commit will halt. Re-running the commit will automatically fix most issues except:
- The flake8 checks... hopefully over time Black (which fixes most things automatically already) will negate need for it.
- You'll have to fix documentation yourself prior to a successful commit (there's no auto fix for that!!).
You can run pre-commit hooks without making a commit, too, like:
pre-commit run black --all-files
or
# -v gives verbose output, useful for figuring out why docs won't build
pre-commit run build-docs -v
Contributing
-
Please raise an issue on the board (or add your $0.02 to an existing issue) so the maintainers know what's happening and can advise / steer you.
-
Create a fork of octue-sdk-python, undertake your changes on a new branch, (see
.pre-commit-config.yaml
for branch naming conventions). To run tests and make commits, you'll need to do something like:
git clone <your_forked_repo_address> # Fetches the repo to your local machine
cd octue-sdk-python # Move into the repo directory
pyenv virtualenv 3.8 myenv # Makes a virtual environment for you to install the dev tools into. Use any python >= 3.8
pyend activate myenv # Activates the virtual environment so you don't screw up other installations
pip install -r requirements-dev.txt # Installs the testing and code formatting utilities
pre-commit install # Installs the pre-commit code formatting hooks in the git repo
-
Adopt a Test Driven Development approach to implementing new features or fixing bugs.
-
Ask the maintainers where to make your pull request. We'll create a version branch, according to the roadmap, into which you can make your PR. We'll help review the changes and improve the PR.
-
Once checks have passed, test coverage of the new code is 100%, documentation is updated and the Review is passed, we'll merge into the version branch.
-
Once all the roadmapped features for that version are done, we'll release.
Release process
The process for creating a new release is as follows:
- Check out a branch for the next version, called
vX.Y.Z
- Create a Pull Request into the
main
branch. - Undertake your changes, committing and pushing to branch
vX.Y.Z
- Ensure that documentation is updated to match changes, and increment the changelog. Pull requests which do not update documentation will be refused.
- Ensure that test coverage is sufficient. Pull requests that decrease test coverage will be refused.
- Ensure code meets style guidelines (pre-commit scripts and flake8 tests will fail otherwise)
- Address Review Comments on the PR
- Ensure the version in
setup.py
is correct and matches the branch version. - Merge to master. Successful test, doc build, flake8 and a new version number will automatically create the release on pypi.
- Go to code > releases and create a new release on GitHub at the same SHA.
Documents
Building documents automatically
The documentation will build automatically in a pre-configured environment when you make a commit.
In fact, the way pre-commit works, you won't be allowed to make the commit unless the documentation builds, this way we avoid getting broken documentation pushed to the main repository on any commit sha, so we can rely on builds working.
Building documents manually
If you did need to build the documentation
Install doxgen
. On a mac, that's brew install doxygen
; other systems may differ.
Install sphinx and other requirements for building the docs:
pip install -r docs/requirements.txt
Run the build process:
sphinx-build -b html docs/source docs/build
Tom Clark, founder of octue We've been developing open-source tools to make it easy for normal, mortal scientists and engineers to easily create, use and connect
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.