Skip to main content

write shell in sh.yml

Project description

Sometimes gnu make reminds me how old I am, shyml makes me feel like a baby again.

Getting started

Install

Install with pip install shyml.

Note

Use pip install --user for non-root install in ~/.local/bin.

Executable and shebang

Create a executable yaml file in your repo (name it sh.yml by convention) with shyml in the shebang as such:

#!/usr/bin/env shyml

Then, start adding a YAML document in it.

Example

#!/usr/bin/env shyml
name: foo
help: bar
requires:
- other
script:
- ./super
     long
     line
- shyml_otherjob     # call another job as a bash function
---
name: otherjob
script: |
  your code here

Document model

Each YAML document (separated by ) should contain a name key.

Other keys it can define:

  • script: a bash script in list or string format, arguments will be proxied
  • help: a help text to describe the job
  • color: a color to render the job name
  • requires: the list of other jobs to execute prior to this job
  • hook: set to before toautomatically execute before any other
  • env: a YAML hash of env var

CLI Usage

./sh.yml                       # lists jobs
./sh.yml jobname               # run a job in a local bash shell
shell=xonsh ./sh.yml jobname   # apparently you your sh.yml contains xonsh instead of bash ^^
./sh.yml debug jobname         # print a job script code
./sh.yml test jobname          # print a job help

Examples

Replace tox.ini

So, initially shyml was born because I wanted to get too much out of tox. Namely, centralizing test automation and eventually deployment (for integration testing) in a single multi-script file, for usage in various contexts:

  • in the system python environment, ie. in a built container
  • in the user python environment, that is where i have checked out all development source code I want to hack at the version that I develop with (and I try hard to stick to upstream and have forward-compatible code)
  • in a virtualenv, to test against released module versions.

To address this, I use such shyml job, that will make a venv with python3 by default, not setup any venv if venv=none, and use the user environment if venv=user.

---
name: install
help: |
  Setup and activate a venv for a python executable

  If venv=none, it will not do any venv.
  If venv=user, it will use pip install --user.
script: |
  if [ "${venv-}" = "user" ]; then
    pip_install="pip install --user"
  elif [ "${venv-}" != "none" ]; then
    export python="${python-python3}"
    export path="${path-.venv.$python}"
    test -d $path || virtualenv --python=$python $path
    set +eux; echo activating $path; source $path/bin/activate; set -eux
  fi
  ${pip_install-pip install} -Ue .[test]

---
name: test
help: Run test in a python3 venv by default.
script: shyml_install && py.test -vv --cov src --strict -r fEsxXw ${@-src}

Then, I can run:

venv=user ./sh.yml test       # in my home
venv=none ./sh.yml test       # in a built container
./sh.yml test                 # just run tests in the default venv tox-like

Embed docker-compose.yml

---
name: compose
script: |
  docker-compose -p $(pwd) -f <(cat <<EOF
  version: '3.5'
  services:
    django:
      build:
        dockerfile: Dockerfile
        context: ./
        shm_size: 512mb
      depends_on:
      - postgres
      volumes:
      - ./:/app

    postgres:
      image: postgres:10
  EOF
  ) $@

Replace Dockerfile with buildah

---
name: buildah
script:
- buildcntr1=$(buildah from --quiet docker.io/node:10-alpine)
- buildmnt1=$(buildah mount $buildcntr1)
- buildah config
    --env DJANGO_SETTINGS_MODULE=mrs.settings
    --env UWSGI_MODULE=mrs.wsgi:application
    --env NODE_ENV=production
    --env PYTHONIOENCODING=UTF-8
    --env PYTHONUNBUFFERED=1
    --env STATIC_URL=/static/
    --env STATIC_ROOT=/app/static
    --env UWSGI_SPOOLER_NAMES=mail,stat
    --env UWSGI_SPOOLER_MOUNT=/app/spooler
    --env VIRTUAL_PROTO=uwsgi
    --env LOG=/app/log
    --env VIRTUAL_PROTO=uwsgi
    --env GIT_COMMIT=$CI_COMMIT_SHA
    $buildcntr1
- mkdir -p .cache/{apk,pip,npm}
- buildah run -v $(pwd)/.cache/apk:/root/.cache/apk $buildcntr1
    apk add
    --update
    --cache-dir /root/.cache/apk
    ca-certificates
    gettext
    shadow
    python3
    py3-pillow
    py3-psycopg2
    dumb-init
    bash
    git
    curl
    uwsgi-python3
    uwsgi-http
    uwsgi-spooler
    uwsgi-cache
    uwsgi-router_cache
    uwsgi-router_static
- buildah -v $(pwd)/.cache/pip:/root/.cache/pip run $buildcntr1
    pip3 install --upgrade pip
- buildah run $buildcntr1
    bash -c 'curl -sL https://sentry.io/get-cli/ | bash'
- buildah run $buildcntr1
    bash -c 'mkdir -p /app && usermod -d /app -l app node && groupmod -n app node && chown -R app:app /app'

- buildah config --workingdir /app $buildcntr1

- buildah copy $buildcntr1
    yarn.lock .babelrc package.json webpack.config.js /app/
- buildah run $buildcntr1
    yarn install --frozen-lockfile
- buildah copy $buildcntr1
    src/mrs/static /app/src/mrs/static
- buildah run $(pwd)/.cache/npm:/npm $buildcntr1
    yarn --cache-folder /npm prepare

- buildah copy $buildcntr1
    requirements.txt /app/
- buildah run -v $(pwd)/.cache/pip:/root/.cache/pip $buildcntr1
    pip3 install --upgrade -r /app/requirements.txt

- buildah run $buildcntr1 bash
- buildah copy $buildcntr1
    setup.py src /app/
- buildah run $buildcntr1 bash
- buildah run -v $(pwd)/.cache/pip:/root/.cache/pip $buildcntr1
    pip3 install --editable /app

- buildah run $buildcntr1
    mkdir -p /app/{log,static}
- buildah run $buildcntr1
    mrs collectstatic --noinput --clear
- buildah copy $buildcntr1
    locale /app/locale
- buildah run $buildcntr1
    mrs compilemessages -l fr
- buildah run $buildcntr1
    'find $STATIC_ROOT -type f | xargs gzip -f -k -9'

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for shyml, version 1.0.3
Filename, size File type Python version Upload date Hashes
Filename, size shyml-1.0.3.tar.gz (6.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page