gaohn-common-utils 0.0.112

A small utility package

Common Utils

Continuous Integration

Virtual Environment

First, make a virtual environment with make_venv.sh:

curl -s -o make_venv.sh \
  https://raw.githubusercontent.com/gao-hongnan/common-utils/main/scripts/devops/make_venv.sh && \
bash make_venv.sh venv --pyproject --dev && \
source venv/bin/activate && \
rm make_venv.sh

Continue on error vs If Always

See here.

Run Bandit Security Check

bash ./scripts/devops/ci/ci_security_bandit.sh \
  --severity-level=low \
  --format=json \
  --output=bandit_results.json \
  common_utils

Run Linter Check

bash ./scripts/devops/ci/ci_linter_pylint.sh \
  --rcfile=pyproject.toml \
  --fail-under=10 \
  --score=yes \
  --output-format=json:pylint_results.json,colorized \
  common_utils

Run Formatter Black Check

bash ./scripts/devops/ci/ci_formatter_black.sh \
  --check \
  --diff \
  --color \
  --verbose \
  common_utils

Run Formatter Isort Check

bash ./scripts/devops/ci/ci_formatter_isort.sh \
  --check \
  --diff \
  --color \
  --verbose \
  common_utils

Run MyPy Type Check

bash ./scripts/devops/ci/ci_typing_mypy.sh \
  --config-file=pyproject.toml \
  common_utils \
  | tee mypy_results.log

Run Unit Test

Run Integration Test

Run System Test

Run Acceptance Test

See madewithml.

Run Data Test (Great Expectations)

Run Markdown Lint

npm install -g markdownlint-cli && \
touch .markdownlint.json && \

npm install --save-dev --save-exact prettier

# prettier
function pr() {
  if [ -z "$1" ]; then
    echo "Error: TARGET_DIR is mandatory."
    return 1
  fi

  TARGET_DIR="$1"
  prettier "$TARGET_DIR" --write \
    --prose-wrap always \
    --print-width 80 \
    --tab-width 4 \
    --use-tabs true
}

pr <TARGET_MARKDOWN_FILE>
markdownlint --fix <TARGET_MARKDOWN_FILE>