No project description provided
Project description
AsyncIO eventloop helpers and Abstract Base Classes for making services that use ZMQ nice, easy and DRY
Usage
Use the CookieCutter template at https://gitlab.com/advian-oss/python-datastreamserviceapp_template
You can also take a look at src/datastreamservicelib/console.py for some very simple test examples.
Docker
For more controlled deployments and to get rid of “works on my computer” -syndrome, we always make sure our software works under docker.
It’s also a quick way to get started with a standard development environment.
SSH agent forwarding
We need buildkit:
export DOCKER_BUILDKIT=1
And also the exact way for forwarding agent to running instance is different on OSX:
export DOCKER_SSHAGENT="-v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock -e SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock"
and Linux:
export DOCKER_SSHAGENT="-v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK -e SSH_AUTH_SOCK"
Creating a development container
Build image, create container and start it (switch the 58449 port to the port from src/datastreamservicelib/defaultconfig.py):
docker build --ssh default --target devel_shell -t datastreamservicelib:devel_shell . docker create --name datastreamservicelib_devel -v `pwd`":/app" -it -v /tmp:/tmp `echo $DOCKER_SSHAGENT` datastreamservicelib:devel_shell docker start -i datastreamservicelib_devel
pre-commit considerations
If working in Docker instead of native env you need to run the pre-commit checks in docker too:
docker exec -i datastreamservicelib_devel /bin/bash -c "pre-commit install" docker exec -i datastreamservicelib_devel /bin/bash -c "pre-commit run --all-files"
You need to have the container running, see above. Or alternatively use the docker run syntax but using the running container is faster:
docker run -it --rm -v `pwd`":/app" datastreamservicelib:devel_shell -c "pre-commit run --all-files"
Test suite
You can use the devel shell to run py.test when doing development, for CI use the “tox” target in the Dockerfile:
docker build --ssh default --target tox -t datastreamservicelib:tox . docker run -it --rm -v `pwd`":/app" `echo $DOCKER_SSHAGENT` datastreamservicelib:tox
Production docker
There’s a “production” target as well for quick running of testsubscriber and testpublisher
docker build –ssh default –target production -t datastreamservicelib:latest . docker run –rm -it -v /tmp:/tmp datastreamservicelib:latest testpublisher -s ipc:///tmp/test_pub.sock -t foo docker run –rm -it -v /tmp:/tmp datastreamservicelib:latest testsubscriber -s ipc:///tmp/test_pub.sock -t foo
Note that on non-linux platforms the IPC sockets may not as expected between host and container over volume mounts.
Local Development
TLDR:
Create and activate a Python 3.6 virtualenv (assuming virtualenvwrapper):
mkvirtualenv -p `which python3.6` my_virtualenv
change to a branch:
git checkout -b my_branch
install Poetry: https://python-poetry.org/docs/#installation
Install project deps and pre-commit hooks:
poetry install pre-commit install pre-commit run --all-files
Ready to go, try the following:
testpublisher --help testsubscriber --help
Use Python 3.6 for development since it’s the lowest supported version so you don’t accidentally use features only available in higher versions and then have to re-do everything when CI tests fail on 3.6.
Remember to activate your virtualenv whenever working on the repo, this is needed because pylint and mypy pre-commit hooks use the “system” python for now (because reasons).
Running “pre-commit run –all-files” and “py.test -v” regularly during development and especially before committing will save you some headache.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for datastreamservicelib-1.9.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 392641c246fb9e71bbd59295c4377b1ff269a9608e419985d9288361ff80ced5 |
|
MD5 | febe26c80ad2ad0de8f20691e90cb14f |
|
BLAKE2b-256 | a67da1e2467ccbb9ba05f58cbcde3eebe95dd7e81d48cd8353664771eb414e14 |
Hashes for datastreamservicelib-1.9.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb77fac5f2df557d575233387632236b1b781b16a7ccc921640572fdfaa735c7 |
|
MD5 | 940df69d44a5ef597c02ecdaae2bf09c |
|
BLAKE2b-256 | 60955a75fa847a065ea8b221d333e1b647194e19dca5b5a349c399c9d7469292 |