Skip to main content

Helpers and utilities for https://gitlab.com/advian-oss/python-datastreamservicelib that are not eventloop specific.

Project description

Core helpers and Abstract Base Classes for making use of ZMQ nice, easy and DRY.

You should probably look at https://gitlab.com/advian-oss/python-datastreamservicelib and https://gitlab.com/advian-oss/python-datastreamserviceapp_template unless you’re working on an adapter for yet unsupported eventloop.

Docker

For more controlled deployments and to get rid of “works on my computer” -syndrome, we always make sure our software works under docker.

It’s also a quick way to get started with a standard development environment.

SSH agent forwarding

We need buildkit:

export DOCKER_BUILDKIT=1

And also the exact way for forwarding agent to running instance is different on OSX:

export DOCKER_SSHAGENT="-v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock -e SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock"

and Linux:

export DOCKER_SSHAGENT="-v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK -e SSH_AUTH_SOCK"

Creating a development container

Build image, create container and start it:

docker build --ssh default --target devel_shell -t datastreamcorelib:devel_shell .
docker create --name datastreamcorelib_devel -v `pwd`":/app" -it -v /tmp:/tmp `echo $DOCKER_SSHAGENT` datastreamcorelib:devel_shell
docker start -i datastreamcorelib_devel

pre-commit considerations

If working in Docker instead of native env you need to run the pre-commit checks in docker too:

docker exec -i datastreamcorelib_devel /bin/bash -c "pre-commit install"
docker exec -i datastreamcorelib_devel /bin/bash -c "pre-commit run --all-files"

You need to have the container running, see above. Or alternatively use the docker run syntax but using the running container is faster:

docker run -it --rm -v `pwd`":/app" datastreamcorelib:devel_shell -c "pre-commit run --all-files"

Test suite

You can use the devel shell to run py.test when doing development, for CI use the “tox” target in the Dockerfile:

docker build --ssh default --target tox -t datastreamcorelib:tox .
docker run -it --rm -v `pwd`":/app" `echo $DOCKER_SSHAGENT` datastreamcorelib:tox

Production docker

There’s a “production” target since it’s our standard pattern, however this library does not yet expose any CLI tools so the production target is kinda useless.

Local Development

TLDR:

  • Create and activate a Python 3.6 virtualenv (assuming virtualenvwrapper):

    mkvirtualenv -p `which python3.6` my_virtualenv
  • change to a branch:

    git checkout -b my_branch
  • install Poetry: https://python-poetry.org/docs/#installation

  • Install project deps and pre-commit hooks:

    poetry install
    pre-commit install
    pre-commit run --all-files
  • Ready to go.

Use Python 3.6 for development since it’s the lowest supported version so you don’t accidentally use features only available in higher versions and then have to re-do everything when CI tests fail on 3.6.

Remember to activate your virtualenv whenever working on the repo, this is needed because pylint and mypy pre-commit hooks use the “system” python for now (because reasons).

Running “pre-commit run –all-files” and “py.test -v” regularly during development and especially before committing will save you some headache.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datastreamcorelib-1.6.0.tar.gz (14.3 kB view details)

Uploaded Source

Built Distribution

datastreamcorelib-1.6.0-py3-none-any.whl (16.2 kB view details)

Uploaded Python 3

File details

Details for the file datastreamcorelib-1.6.0.tar.gz.

File metadata

  • Download URL: datastreamcorelib-1.6.0.tar.gz
  • Upload date:
  • Size: 14.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.16

File hashes

Hashes for datastreamcorelib-1.6.0.tar.gz
Algorithm Hash digest
SHA256 891032defbc50836bd1af7a6c70132bfb134559678da1ca7dfd8de20b1d13739
MD5 e1ebc8075e004fec9bc222654728a74b
BLAKE2b-256 99f2e29d3332dcf5f1518ae3a9a3de2618d47f9c9084edb57dc28b3d9cd1611a

See more details on using hashes here.

File details

Details for the file datastreamcorelib-1.6.0-py3-none-any.whl.

File metadata

File hashes

Hashes for datastreamcorelib-1.6.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1f456d0d284112dbd16d4eb9bbe95391e44163e3b98eff6699e8ab521f168855
MD5 a2377909790b7ad7951d11b70ba97f57
BLAKE2b-256 ea51b134ff18269d24c985d678ed1f29c6c8d209127aa26a337ed1b71d87c98f

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page