travis related utilities
Project description
lib_travis
Version v2.3.12 as of 2021-02-15 see Changelog
- small utils for travis:
print colored banners
wrap commands into run/success/error banners, with automatic retry
resolve the branch to test, based on the travis environment variables
automated tests, Travis Matrix, Documentation, Badges, etc. are managed with PizzaCutter (cookiecutter on steroids)
Python version required: 3.6.0 or newer
tested on linux “bionic” with python 3.6, 3.7, 3.8, 3.9, 3.9-dev, pypy3 - architectures: amd64, ppc64le, s390x, arm64
100% code coverage, flake8 style checking ,mypy static type checking ,tested under Linux, macOS, automatic daily builds and monitoring
Try it Online
You might try it right away in Jupyter Notebook by using the “launch binder” badge, or click here
Usage
usage commandline:
Usage: lib_travis [OPTIONS] COMMAND [ARGS]...
travis related utilities
Options:
--version Show the version and exit.
--traceback / --no-traceback return traceback information on cli
-h, --help Show this message and exit.
Commands:
install
updates pip, setuptools, wheel, pytest-pycodestyle
--dry-run
script
updates pip, setuptools, wheel, pytest-pycodestyle
--dry-run
after_success
coverage reports
--dry-run
deploy
deploy on pypi
--dry-run
get_branch
get the branch to work on
info
get program informations
run [Options] <description> <command>
run string command wrapped in run/success/error banners
-r --retry retry n times, default = 3
-s --sleep sleep when retry, default = 30 seconds
--banner --no-banner wrap in banners, default = True
run a command passed as string
# to be used in travis.yml
# run a command passed as string, wrap it in colored banners, retry 3 times, sleep 30 seconds when retry
$> lib_travis run "description" "command -some -options" --retry=3 --sleep=30
get the branch to work on from travis environment variables
$> BRANCH=$(lib_travis get_branch)
python methods:
install, jobs to do in the Travis “install” section
def install(dry_run: bool = True) -> None:
"""
upgrades pip, setuptools, wheel and pytest-pycodestyle
Parameter
---------
cPIP
from environment, the command to launch pip, like "python -m pip"
Examples
--------
>>> if os.getenv('TRAVIS'):
... install(dry_run=True)
"""
script, jobs to do in the Travis “script” section
def script(dry_run: bool = True) -> None:
"""
travis jobs to run in travis.yml section "script":
- run setup.py test
- run pip with install option test
- run pip standard install
- test the CLI Registration
- install the test requirements
- install codecov
- install pytest-codecov
- run pytest coverage
- run mypy strict
- if MYPY_STRICT="True"
- rebuild the rst files (resolve rst file includes)
- needs RST_INCLUDE_SOURCE, RST_INCLUDE_TARGET set and BUILD_DOCS="True"
- check if deployment would succeed, if setup.py exists and not a tagged build
Parameter
---------
cPREFIX
from environment, the command prefix like 'wine' or ''
cPIP
from environment, the command to launch pip, like "python -m pip"
cPYTHON
from environment, the command to launch python, like 'python' or 'python3' on MacOS
CLI_COMMAND
from environment, must be set in travis - the CLI command to test with option --version
MYPY_STRICT
from environment, if pytest with mypy --strict should run
PACKAGE_NAME
from environment, the package name to pass to mypy
BUILD_DOCS
from environment, if rst file should be rebuilt
RST_INCLUDE_SOURCE
from environment, the rst template with rst includes to resolve
RST_INCLUDE_TARGET
from environment, the rst target file
DEPLOY_WHEEL
from environment, if a wheel should be generated
only if setup.py exists and on non-tagged builds (there we deploy for real)
dry_run
if set, this returns immediately - for CLI tests
Examples
--------
>>> script()
"""
after_success, jobs to do in the Travis “after_success” section
def after_success(dry_run: bool = True) -> None:
"""
travis jobs to run in travis.yml section "after_success":
- coverage report
- codecov
- codeclimate report upload
Parameter
---------
cPREFIX
from environment, the command prefix like 'wine' or ''
cPIP
from environment, the command to launch pip, like "python -m pip"
CC_TEST_REPORTER_ID
from environment, must be set in travis
TRAVIS_TEST_RESULT
from environment, this is set by TRAVIS automatically
dry_run
if set, this returns immediately - for CLI tests
Examples
--------
>>> after_success()
"""
deploy, deploy to pypi in the Travis “after_success” section
def deploy(dry_run: bool = True) -> None:
"""
uploads sdist and wheels to pypi on success
Parameter
---------
cPREFIX
from environment, the command prefix like 'wine' or ''
PYPI_PASSWORD
from environment, passed as secure, encrypted variable to environment
TRAVIS_TAG
from environment, needs to be set
DEPLOY_SDIST, DEPLOY_WHEEL
from environment, one of it needs to be true
dry_run
if set, this returns immediately - for CLI tests
Examples
--------
>>> deploy()
"""
get_branch, determine the branch to work on from Travis environment
def get_branch() -> str:
"""
Return the branch to work on
Parameter
---------
TRAVIS_BRANCH
from environment
TRAVIS_PULL_REQUEST_BRANCH
from environment
TRAVIS_TAG
from environment
Result
---------
the branch
Exceptions
------------
none
============ ============= ========================== ========== =======================================================
Build TRAVIS_BRANCH TRAVIS_PULL_REQUEST_BRANCH TRAVIS_TAG Notice
============ ============= ========================== ========== =======================================================
Custom Build <branch> --- --- return <branch> from TRAVIS_BRANCH
Push <branch> --- --- return <branch> from TRAVIS_BRANCH
Pull Request <master> <branch> --- return <branch> from TRAVIS_PULL_REQUEST_BRANCH
Tagged <tag> --- <tag> return 'master'
============ ============= ========================== ========== =======================================================
TRAVIS_BRANCH:
for push builds, or builds not triggered by a pull request, this is the name of the branch.
for builds triggered by a pull request this is the name of the branch targeted by the pull request.
for builds triggered by a tag, this is the same as the name of the tag (TRAVIS_TAG).
Note that for tags, git does not store the branch from which a commit was tagged. (so we use always master in that case)
TRAVIS_PULL_REQUEST_BRANCH:
if the current job is a pull request, the name of the branch from which the PR originated.
if the current job is a push build, this variable is empty ("").
TRAVIS_TAG:
If the current build is for a git tag, this variable is set to the tag’s name, otherwise it is empty ("").
"""
run, usually used internally
def run(
description: str,
command: str,
retry: int = 3,
sleep: int = 30,
banner: bool = True,
show_command: bool = True,
) -> None:
"""
runs and retries a command passed as string and wrap it in "success" or "error" banners
Parameter
---------
description
description of the action, shown in the banner
command
the command to launch
retry
retry the command n times, default = 3
sleep
sleep for n seconds between the commands, default = 30
banner
if to use banner for run/success or just colored lines.
Errors will be always shown as banner
show_command
if the command is shown - take care not to reveal secrets here !
Result
---------
none
Exceptions
------------
none
Examples
------------
>>> run('test', "unknown command", sleep=0)
Traceback (most recent call last):
...
SystemExit: ...
>>> run('test', "unknown command", sleep=0, show_command=False)
Traceback (most recent call last):
...
SystemExit: ...
>>> run('test', "echo test")
>>> run('test', "echo test", show_command=False)
"""
travis.py example
language: python
group: travis_latest
dist: bionic
sudo: true
env:
global:
# prefix before commands - used for wine, there the prefix is "wine"
- cPREFIX=""
# command to launch python interpreter (its different on macOs, there we need python3)
- cPYTHON="python"
# command to launch pip (its different on macOs, there we need pip3)
- cPIP="python -m pip"
# switch off wine fix me messages
- WINEDEBUG=fixme-all
# PYTEST
- PYTEST_DO_TESTS="True"
# FLAKE8 tests
- DO_FLAKE8_TESTS="True"
# MYPY tests
- MYPY_DO_TESTS="True"
- MYPY_OPTIONS="--follow-imports=normal --implicit-reexport --no-warn-unused-ignores --strict"
- MYPYPATH="./lib_travis/3rd_party_stubs"
# coverage
- DO_COVERAGE="True"
- DO_COVERAGE_UPLOAD_CODECOV="True"
- DO_COVERAGE_UPLOAD_CODE_CLIMATE="True"
# package name
- PACKAGE_NAME="lib_travis"
# the registered CLI Command
- CLI_COMMAND="lib_travis"
# the source file for rst_include (rebuild rst file includes)
- RST_INCLUDE_SOURCE="./.docs/README_template.rst"
# the target file for rst_include (rebuild rst file includes)
- RST_INCLUDE_TARGET="./README.rst"
# secure environment variable
- secure: "O8oOuJ4NfK9HRstY43GSgZcvmR2K/3N+rWzzNYHgKoNvOeBUMMQ6t90Pmgir/VfBF6TvkMxJwXnkAvFCtFAvzRN6KdgVq9UqkJ7CCS9woQxddPfFoTNwQDrySWmbD+cefh1Dt2uIGbnIhvWVCWlYbNAxfZ3IOTEhqfl2OOza44P8PJPplp83wKgbD899Th3qY8k77Kzh9+B2ErFVDBtVe8M10vI74WkuyZ/HP4ue1z5noa9cklkm9B1stf1Jqf4vW4lGnnsaR1camM5PH03OpFquxOvTTaoI+FAntW8zl8C+sFLC1xp4Emu3lT73OFtEXYUsvI7InWFEwl5k4pMMAXFVE4Nqf5PZviXGxtOTNI13gSBVcMw44eujoRpVrXzixQdrbhYUcID9mC3ApRh2002XxGVvHgLXxiKkBG/1yMNCsuRZrev45j4J/GItNvfswEtLDUTWOO+plxWsNSh2V7qfHFLnNJyBCU8sGj7sgyzQvS4YyJQ5V7pMeoXyvwnOU1lHYRCXbshO6Y/r3+PkCr1UcQXwH6/1awwwNDGK4DJ3BZHGeJTr7NI/IiJvgnLnXUAN6hpNW1V3GEbPMkz3FmcPkivmvVD8XOBl+w8OLghGCyqETbIfFhvplZ/S3dBthJLxlpEqJItNeG5dwdLAytwpFLWu40biRzrf/GsjkS8=" # CC_TEST_REPORTER_ID.secret
- secure: "ME0CLFezdNlt8gbWEXNq6c8uT2kzbLmp4GoxUlJpsYG8A3xV+KrDXJ/RR66So62A5tvdxd8d8IN7j3HcapXqMyi/mbCDVu2eZXKOl/cGphMUS5lI80jjs/Yp6CwNC/qvBhe6bqkOXvhZS4I+4GTJOtRUEQJ5OLrf0uoGEVq9Tx6WKAElgcYMXdJNKwW8hSfUg5N2Ujq/PaaMXj7vIJSUBKWioIIx5doF7YSlixwCxj0HStyxX/lSy6VO1FJO2ZFsNN/jglduBJzbrYh6gcUbUc6fHCKzlr4RM70Ylb1rRkZSSY1uDvgTM6WiUS/LwN+65v+Z/rbucPJ/l8SGoEyrS/xytcECUq1oiyq+IMvJplspCV3/m+UGYd8jdJsG+O+WQQE0PknhBAtp22HJgI8xjU67SArXtbxA0/WhN/R7hulphnCDCqkyjR8lsao6mjXligsGbvHdm17u6dm0OCgpjA+R10ZW4SCOwDuGyA0prqOCsODRYLNnGXrNHUhJvrmx1TgkrIPL5VJIW3urujIj/G+clCgPzb+Dr/VAys0FNaRug1qFaoGAEFDutSJ/cALfTdiIU3RAVJ8lIChxmnaGneDf7GUKo36jjs9rdc5cqjHX4BG/7t5XSIymS8tBx1CWTxYVIGEhzSswDwcT2iZhBEuj+AJnY5ThE+bVNg2zhDw=" # PYPI_PASSWORD.secret
addons:
apt:
packages:
- xvfb # install xvfb virtual framebuffer - this we need for WINE
- winbind # needed for WINE
services: # start services
- xvfb # is needed for WINE on headless installation
matrix:
include:
- os: linux
arch: "amd64"
if: true
language: python
python: "3.6"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: linux
arch: "amd64"
if: true
language: python
python: "3.7"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="False"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: linux
arch: "amd64"
if: true
language: python
python: "3.8"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: linux
arch: "amd64"
if: true
language: python
python: "3.9"
before_install:
- export BUILD_DOCS="True"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: linux
arch: "amd64"
if: true
language: python
python: "3.9-dev"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: linux
arch: "amd64"
if: true
language: python
python: "pypy3"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="False"
- os: linux
arch: "ppc64le"
if: tag IS present
language: python
python: "3.9"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: linux
arch: "s390x"
if: tag IS present
language: python
python: "3.9"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: linux
arch: "arm64"
if: tag IS present
language: python
python: "3.9"
before_install:
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="True"
- export DEPLOY_WHEEL="True"
- export BUILD_TEST="True"
- export MYPY_DO_TESTS="True"
- os: osx
if: true
language: sh
name: "macOS 10.15.7"
python: "3.8"
osx_image: xcode12.2
env:
# on osx pip and python points to python 2.7 - therefore we have to use pip3 and python3 here
- cPREFIX="" # prefix before commands - used for wine, there the prefix is "wine"
- cPYTHON="python3" # command to launch python interpreter (its different on macOs, there we need python3)
- cPIP="python3 -m pip" # command to launch pip (its different on macOs, there we need pip3)
- export BUILD_DOCS="False"
- export DEPLOY_SDIST="False"
- export DEPLOY_WHEEL="False"
- export DEPLOY_TEST="True"
- export MYPY_DO_TESTS="True"
install:
- ${cPIP} install lib_travis
- log_util --colortest
- lib_travis install
script:
- BRANCH=$(lib_travis get_branch)
- log_util --level=NOTICE --banner "working on branch ${BRANCH}"
- lib_travis script
after_success:
- lib_travis after_success
- lib_travis deploy
- ls -l ./dist
notifications:
email:
recipients:
- bitranox@gmail.com
# on_success default: change
on_success: never
on_failure: always
Usage from Commandline
Usage: lib_travis [OPTIONS] COMMAND [ARGS]...
travis related utilities
Options:
--version Show the version and exit.
--traceback / --no-traceback return traceback information on cli
-h, --help Show this message and exit.
Commands:
after_success coverage reports
deploy deploy on pypi
get_branch get the branch to work on
info get program informations
install updates pip, setuptools, wheel, pytest-pycodestyle
run run string command wrapped in run/success/error banners
script updates pip, setuptools, wheel, pytest-pycodestyle
Installation and Upgrade
Before You start, its highly recommended to update pip and setup tools:
python -m pip --upgrade pip
python -m pip --upgrade setuptools
to install the latest release from PyPi via pip (recommended):
python -m pip install --upgrade lib_travis
to install the latest version from github via pip:
python -m pip install --upgrade git+https://github.com/bitranox/lib_travis.git
include it into Your requirements.txt:
# Insert following line in Your requirements.txt:
# for the latest Release on pypi:
lib_travis
# for the latest development version :
lib_travis @ git+https://github.com/bitranox/lib_travis.git
# to install and upgrade all modules mentioned in requirements.txt:
python -m pip install --upgrade -r /<path>/requirements.txt
to install the latest development version from source code:
# cd ~
$ git clone https://github.com/bitranox/lib_travis.git
$ cd lib_travis
python setup.py install
via makefile: makefiles are a very convenient way to install. Here we can do much more, like installing virtual environments, clean caches and so on.
# from Your shell's homedirectory:
$ git clone https://github.com/bitranox/lib_travis.git
$ cd lib_travis
# to run the tests:
$ make test
# to install the package
$ make install
# to clean the package
$ make clean
# uninstall the package
$ make uninstall
Requirements
following modules will be automatically installed :
## Project Requirements
click
cli_exit_tools
lib_log_utils
rst_include
Acknowledgements
special thanks to “uncle bob” Robert C. Martin, especially for his books on “clean code” and “clean architecture”
Contribute
I would love for you to fork and send me pull request for this project. - please Contribute
License
This software is licensed under the MIT license
—
Changelog
new MAJOR version for incompatible API changes,
new MINOR version for added functionality in a backwards compatible manner
new PATCH version for backwards compatible bug fixes
v2.3.12
- 2021-02-15: service release
s390x, arm and ppc to python 3.9
v2.3.11
- 2021-02-15: service release
update macOS to 10.15.7
v2.3.10
- 2021-02-15: service release
disable rust-crypto build for architectures s390x and ppc64le, since compile errors
v2.3.9
- 2021-02-15: service release
install rust compiler for pypy3 on linux, needed for twine
update secrets for travis-ci.com
v2.3.8
- 2021-02-15: service release
install rust compiler for pypy3 on linux, needed for twine
v2.3.7
- 2020-10-09: service release
update travis build matrix for linux 3.9-dev
update travis build matrix (paths) for windows 3.9 / 3.10
v2.3.6
- 2020-08-16: service release
correct requirements
v2.3.5
- 2020-08-15: service release
install requirements
more cleanup
black codestyle
v2.3.4
- 2020-08-08: service release
cleanup
v2.3.3
- 2020-08-08: service release
fix documentation
fix travis
deprecate pycodestyle
implement flake8
v2.3.2
- 2020-08-07:
fix pypy eggs
v2.3.1
- 2020-08-07:
fix mypy pypy
v2.3.0
- 2020-08-07:
mypy without pytest
coverage refractor
v2.2.0
2020-08-06: integrate flake8 (in transition)
v2.1.0
2020-08-06: integrate flake8 (in transition)
v2.0.9
2020-08-01: fix environment namings
v2.0.8
2020-08-01: fix pypi deploy
v1.0.6
- 2020-07-29: feature release
preserve Travis Tag
v1.0.5
- 2020-07-29: feature release
pass correct package name to mypy and codecov
v1.0.4
- 2020-07-29: feature release
use the new pizzacutter template
use cli_exit_tools
v1.0.3
- 2020-07-29: feature release
fix code coverage test report
v1.0.2
- 2020-07-29: feature release
do not reveal secrets in error messages
v1.0.1
- 2020-07-29: feature release
documentation updates
v1.0.0
2020-07-29: Release 1.0.0 fully functional
v0.4.9
- 2020-07-27: feature release
add command script
add command after_success
add command deploy
v0.4.8
2020-07-27: debug
v0.4.7
2020-07-27: debug
v0.4.6
2020-07-27: debug
v0.4.5
- 2020-07-27: feature release
add command run_tests
v0.4.3
- 2020-07-27: feature release
set default to –no-split on run_x
add command upgrade_setup_related
v0.4.2
- 2020-07-27: feature release
change colors
catch all in run exceptions (OS Error)
v0.4.1
- 2020-07-27: feature release
use cli_exit_tools
adding banner parameter to “run” commands
v0.4.0
- 2020-07-23: feature release
rename commands
v0.3.1
- 2020-07-23: feature release
add splitting of commands
v0.3.0
- 2020-07-23: feature release
add second run method
add automatic quoting for commands passed as string
v0.2.1
- 2020-07-23: patch release
flush streams on exit
v0.2.0
- 2020-07-23: feature release
change arguments
add options for retry and sleep on run command
v0.1.3
- 2020-07-23: patch release
correct doctests
v0.1.2
- 2020-07-23: patch release
ignore unused options on cli run command
added description argument to run command
v0.1.1
- 2020-07-23: initial release
setup
log utils
run wrapper
get the branch to work on
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for lib_travis-2.3.12-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6cd82a9a23545291acee8aa51a9196608d60e8c3a39a2ce046fb2d3abfe755dc |
|
MD5 | 9ea4b552a680340a77966b94518292a9 |
|
BLAKE2b-256 | c6e9a70014374e35db65f66ace94287b77622ee609af76ac925e4dbd9f70baa3 |