dlunch 2.9.0

The ultimate web app for a well organized lunch.

Data Lunch

The ultimate web app for a well organized lunch.

1. Table of contents

• 1. Table of contents
• 2. Development environment setup
  • 2.1. Miniconda
  • 2.2. Environment variables
    • 2.2.1. General
    • 2.2.2. Docker and Google Cloud Platform
    • 2.2.3. TLS/SSL Certificate
    • 2.2.4. Encryption and Authorization
  • 2.3. Setup the development environment
  • 2.4. Manually install data-lunch CLI
  • 2.5. Running the docker-compose system
  • 2.6. Running a single container
• 3. Additional installations before contributing
  • 3.1. Pre-commit hooks
  • 3.2. Commitizen
• 4. Release strategy from development to main branch
• 5. Google Cloud Platform utilities

2. Development environment setup

The following steps will guide you through the installation procedure.

2.1. Miniconda

is required for creating the development environment (it is suggested to install Miniconda).

Use the terminal for navigating to the repository base directory.
Use the following command in your terminal to create an environment named data-lunch manually.
Otherwise use the setup script to activate the guided installing procedure.

conda env create -f environment.yml

Activate the new Conda environment with the following command.

conda activate data-lunch

2.2. Environment variables

The following environment variables are required for running the web app, the makefile or utility scripts.

2.2.1. General

Variable	Type	Description
PANEL_APP	str	app name, data-lunch-app by default (used by makefile)
PANEL_ENV	str	environment, e.g. development, quality, production
PANEL_ARGS	str	additional arguments passed to Hydra (e.g. panel/gui=major_release)
PORT	int	port used bu the web app (or the container), default to 5000

2.2.2. Docker and Google Cloud Platform

Variable	Type	Description
DOCKER_USERNAME	str	your Docker Hub username, used by makefile and stats panel to extract container name (optional)
IMAGE_VERSION	str	Docker image version, typically stable or latest
GCLOUD_PROJECT	str	Google Cloud Platform project_id, used by makefile for GCP's CLI authentication and for uploading the database to gcp storage, if active in web app configuration files (see panel.scheduled_tasks)
GCLOUD_BUCKET	str	Google Cloud Platform bucket, used for uploading database to gcp storage, if active in web app configuration files (see panel.scheduled_tasks)
MAIL_USER	str	email client user, used for sending emails containing the instance IP, [e-mail] (used only for Google Cloud Platform)
MAIL_APP_PASSWORD	str	email client password used for sending emails containing the instance IP (used only for Google Cloud Platform)
MAIL_RECIPIENTS	str	email recipients as string, separated by , (used for sending emails containing the instance IP when hosted on Google Cloud Platform)
DUCKDNS_URL	str	URL used in compose_init.sh to update dynamic address (see Duck DNS's instructions for details, used when hosted on Google Cloud Platform)

2.2.3. TLS/SSL Certificate

Variable	Type	Description
CERT_EMAIL	str	email for registering SSL certificates, shared with the authority Let's Encrypt (via certbot)
DOMAIN	str	domain name, e.g. mywebapp.com

2.2.4. Encryption and Authorization

Variable	Type	Description
DATA_LUNCH_COOKIE_SECRET	str	Secret used for securing the authentication cookie (use make generate-secrets to generate a valid secret)
DATA_LUNCH_OAUTH_ENC_KEY	str	Encription key used by the OAuth algorithm for encryption (use make generate-secrets to generate a valid secret)
DATA_LUNCH_OAUTH_KEY	str	OAuth key used for configuring the OAuth provider (GitHub, Azure)
DATA_LUNCH_OAUTH_SECRET	str	OAuth secret used for configuring the OAuth provider (GitHub, Azure)
DATA_LUNCH_OAUTH_REDIRECT_URI	str	OAuth redirect uri used for configuring the OAuth provider (GitHub, Azure), do not set to use default value
DATA_LUNCH_OAUTH_TENANT_ID	str	OAuth tenant id used for configuring the OAuth provider (Azure), do not set to use default value
DATA_LUNCH_DB_USER	str	Postgresql user, do not set to use default value
DATA_LUNCH_DB_PASSWORD	str	Postgresql password
DATA_LUNCH_DB_HOST	str	Postgresql host, do not set to use default value
DATA_LUNCH_DB_PORT	str	Postgresql port, do not set to use default value
DATA_LUNCH_DB_DATABASE	str	Postgresql database, do not set to use default value
DATA_LUNCH_DB_SCHEMA	str	Postgresql schema, do not set to use default value

2.3. Setup the development environment

Use the setup script (setup_dev_env.sh) to install all the required development tools.

Use source to properly launch the script.

source setup_dev_env.sh

2.4. Manually install data-lunch CLI

This step is not required if the setup script is used.

The CLI is distributed with setuptools instead of using Unix shebangs.
It is a very simple utility to initialize and delete the app database. There are different use cases:

• Create/delete the sqlite database used by the app
• Initialize/drop tables inside the sqlite database

Use the following command for generating the CLI executable from the setup.py file, it will install your package locally.

pip install .

If you want to make some changes to the source code it is suggested to use the following option.

pip install --editable .

It will just link the package to the original location, basically meaning any changes to the original package would reflect directly in your environment.

Now you can activate the Conda environment and access the CLI commands directly from the terminal (without using annoying shebangs or prepending python to run your CLI calls).

Test that everything is working correctly with the following commands.

data-lunch --version
data-lunch --help

2.5. Running the docker-compose system

Since this app will be deployed with an hosting service a Dockerfile to build a container image is available.
The docker compose file (see docker-compose.yaml) builds the web app container along with a load balancer (the nginx container) to improve the system scalability.

Look inside the makefile to see the docker and docker-compose options.

To build and run the dockerized system you have to install Docker.
Call the following make command to start the build process.

make up-init up-build

up-init initialize the ssl certificate based on the selected environment (as set in the environment variable PANEL_ENV, i.e. development or production).
Call only make (without arguments) to trigger the same command.
Not initializing ssl certificate folders will result in an nginx container failure on start-up.

The two images are built and the two containers are started.

You can then access your web app at http://localhost:4000.

Note:
You can also use make up to spin up the containers if you do not need to re-build any image or initialize ssl certificate folders.

2.6. Running a single container

It is possible to launch a single server by calling the following command.

make build

make run

You can then access your web app at http://localhost:5000 (if the deafult PORT is selected).

3. Additional installations before contributing

This step is not required if the setup script is used.

Before contributing please create the pre-commit and commitizen environments.

cd requirements
conda env create -f pre-commit.yml
conda env create -f commitizen.yml

3.1. Pre-commit hooks

This step is not required if the setup script is used.

Then install the precommit hooks.

conda activate pre-commit
pre-commit install
pre-commit autoupdate

Optionally run hooks on all files.

pre-commit run --all-files

3.2. Commitizen

This step is not required if the setup script is used.

The Commitizen hook checks that rules for conventional commits are respected in commits messages. Use the following command to enjoy Commitizen's interactive prompt.

conda activate commitizen
cz commit

cz c is a shorther alias for cz commit.

4. Release strategy from development to main branch

This step is required only if the CI-CD pipeline on GitHub does not work.

In order to take advantage of Commitizen bump command follow this guideline.

First check that you are on the correct branch.

git checkout main

Then start the merge process forcing it to stop before commit (--no-commit) and without using the fast forward strategy (--no-ff).

git merge development --no-commit --no-ff

Check that results match your expectations then commit (you can leave the default message).

git commit

Now Commitizen bump command will add an additional commit with updated versions to every file listed inside pyproject.toml.

cz bump --no-verify

You can now merge results of the release process back to the development branch.

git checkout development
git merge main --no-ff

Use "update files from last release" or the default text as commit message.

5. Google Cloud Platform utilities

This step is not required if the setup script is used.

The makefile has two rules for conviniently setting up and removing authentication credentials for Google Cloud Platform command line interface: gcp-config and gcp-revoke.