leaf-portal 1.0.16

LEAF Portal

leaf-portal

Web portal for the LEAF framework. Built with NiceGUI and asyncpg, backed by a TimescaleDB/PostgreSQL database.

Features

• Organisation & department management — hierarchical grouping of entities
• User management — superadmin, org admin, and regular users with bcrypt-hashed passwords; admin impersonation
• Access management — time-windowed grants per department/entity
• Entity management — hide entities from regular users and the sensor catalog
• Sensor data explorer — browse and filter readings by entity, metric, and time range (multi-select)
• Interactive plots — Plotly-based time-series visualization
• Alarm rules — threshold-based alerts (configurable per-rule check interval) with email notifications on trigger and auto-resolve
• API token management — generate and revoke tokens for REST API access
• REST API — token-authenticated endpoints for sensor data retrieval (Swagger UI at /api/docs)
• First-run setup wizard — browser-based DB connection and superadmin creation at /setup
• Password reset — email-based reset flow (/forgot-password, /reset-password)

Requirements

• Python 3.12+
• PostgreSQL 16+ or TimescaleDB
• SMTP server (optional — required for alarm emails and password reset)

Installation

From PyPI:

pip install leaf-portal

From source:

poetry install

Configuration

Create a .env file in the working directory. All variables are optional at startup — the setup wizard at /setup will prompt for DB credentials on first run and persist them to .env.

# Database (defaults shown)
PGHOST=timescaledb
PGPORT=5432
PGDATABASE=leaf
PGUSER=postgres
PGPASSWORD=

# Mail (required for alarm emails and password reset)
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USER=[e-mail]
SMTP_PASSWORD=secret
SMTP_FROM=[e-mail]

# Portal public URL (used in password-reset emails)
PORTAL_URL=http://localhost:8081

# NiceGUI session secret — change in production
STORAGE_SECRET=change-me-in-production

Running

leaf-portal
# or
python -m leaf_portal

The portal listens on 0.0.0.0:8081 by default.

On first run, navigate to http://localhost:8081 — you will be redirected to the setup wizard to configure the database connection and create the initial superadmin account.

REST API

All endpoints require a token passed via the Authorization: Bearer <token> header.

Tokens are generated from the API Tokens page (/tokens).

Method	Path	Description
GET	/api/managements	List managements accessible to the token
GET	/api/data/recent	Most recent readings across all accessible departments (?limit=20)
GET	/api/data	Filtered sensor data — requires organisation + department; optional: entity, metric (comma-separated for multiple), from, to (ISO 8601), limit (max 10 000)

Interactive docs: /api/docs

Development

poetry install --with dev

# Run tests
poetry run pytest tests/ -v

# Lint and format
poetry run ruff check leaf_portal/ tests/
poetry run ruff format leaf_portal/ tests/

Database

The schema lives at deploy/deploy.sql (targets TimescaleDB) and is applied automatically by the setup wizard or on startup when the DB is reachable.

For CI and Docker-based local development, docker/init_db.py waits for Postgres to be ready and then applies deploy/deploy.sql.

Backups

A full pg_dump of the database doesn't make sense once sensor_data grows large — it would re-export the entire sensor history every day. Instead, docker/backup builds a small standalone image that splits the backup in two:

1. Operational tables (organisation, department, user_account, management, alarm_*, mapper_*, api_token, ...) — these are tiny, so they're fully pg_dump'd (custom format) every day to leaf_YYYY-MM-DD.dump. sensor_data and TimescaleDB's internal chunk/catalog tables are excluded.
2. sensor_data — exported per UTC day via \copy to sensor_data_YYYY-MM-DD.csv.gz. The last ROLLING_DAYS days are re-exported (overwritten) on every run to catch late-arriving readings; older files are write-once.

The image is read-only against the database (leaf_backup_user, member of the backup_readers role created by deploy/deploy.sql) and runs once per invocation — schedule it with a Kubernetes CronJob (see docker/backup/cronjob.example.yaml) or any host cron running docker run.

docker build -t leaf-backup docker/backup

docker run --rm \
  -e PGHOST=... -e PGUSER=leaf_backup_user -e PGPASSWORD=... -e PGDATABASE=leaf \
  -e ROLLING_DAYS=3 -e KEEP_DUMPS=14 \
  -v /path/to/backups:/backups \
  leaf-backup

Or, using deploy/deploy.py (builds/pushes via build-backup, runs once via backup):

python3 deploy/deploy.py build-backup   # build & push to the registry

export PGHOST=... PGUSER=leaf_backup_user PGPASSWORD=... PGDATABASE=leaf
export BACKUP_DIR=/path/to/backups      # default: ./backups
python3 deploy/deploy.py backup

Restore:

# 1. Recreate the schema (also recreates the sensor_data hypertable)
python3 deploy/deploy.py schema

# 2. Restore operational tables
pg_restore --data-only --disable-triggers -d <db> leaf_YYYY-MM-DD.dump

# 3. Re-import sensor_data for each day
zcat sensor_data_YYYY-MM-DD.csv.gz | psql -d <db> -c "\copy sensor_data FROM STDIN WITH (FORMAT csv, HEADER true)"

The backup image pins its pg_dump/pg_restore version to the TimescaleDB version this project targets (timescale/timescaledb:2.17.2-pg16) — custom-format dumps aren't readable by an older pg_restore. To back up a different Postgres major version, change the FROM tag in docker/backup/Dockerfile and rebuild. Before dumping anything, backup.sh checks that the server's major version matches its bundled pg_dump and exits with an error (without writing any files) if they've drifted apart.

Deployment

deploy/deploy.py is a helper script for building and running the portal in production. It requires no extra dependencies beyond Docker (and psql for remote schema application).

python3 deploy/deploy.py <command>

Command	What it does
build	Builds a multi-arch (amd64/arm64) Docker image, tags it with the current git tag or short commit hash, and pushes it to docker-registry.wur.nl/leaf/docker/leaf-portal.
schema	Applies deploy/deploy.sql to the target database. Locally it runs psql inside the timescaledb container; against a remote host it calls psql directly.
run	Pulls the portal image from the registry and starts it as a container named leaf-portal on port 8081.
stop	Stops and removes the leaf-portal container.

schema requires passwords for the PostgreSQL service accounts it creates — never use defaults:

export LEAF_PORTAL_PASSWORD=...   # leaf_portal_user  (portal, read+write)
export LEAF_GRAFANA_PASSWORD=...  # leaf_grafana_user (Grafana, read-only)
export LEAF_API_PASSWORD=...      # leaf_api_user     (external API access)
export LEAF_BACKUP_PASSWORD=...   # leaf_backup_user  (backup job, read-only)

Typical production flow:

# 1. Apply the schema (once, or after schema changes)
python3 deploy/deploy.py schema

# 2. Start the portal
python3 deploy/deploy.py run

Run build only when cutting a new release.

Page routes

Route	Description
/	Redirects to /dashboard or /login
/login	Login page
/setup	First-run setup wizard
/forgot-password	Password reset request
/reset-password	Password reset with token
/dashboard	Overview dashboard
/admin/organisations	Organisation management
/admin/departments	Department management
/admin/users	User management
/admin/access-management	Access grant management
/admin/mapper	Entity/metric mapping
/admin/settings	Application settings
/dept/members	Department member management
/entities	Entity management (hide/show from regular users)
/categories	Category management
/data/explorer	Sensor data explorer
/data/plots	Time-series plots
/alarms	Alarm rules and event history
/tokens	API token management
/profile	User profile
/api/docs	Swagger UI for the REST API