undertow

Python client and command line tool for a Trident dataspace server: files, services, offerings, compute jobs and their results, master data and provisioning.

Install

pip install trident-undertow    # the command and the Python package are `undertow`
pip install "trident-undertow[hdf5]"   # adds h5py for reading HDF5 metadata (files inspect, upload, sync)

Python 3.10 or newer.

Configure

Settings are read from the command line options, then the environment (also from a .env file in the working directory, see .env.example), then ~/.undertow/credentials.

The options --url, --username/-u, --password/-p, --auth, --output/-o table|json|yaml and --log/-l come before the command, e.g. undertow -o json files list.

~/.undertow/ holds credentials (written by system login), tokens.json (device flow) and optionally secrets.yaml. All files are created with mode 0600.

Log in

There are three ways to get a token, chosen with UNDERTOW_AUTH or --auth:

• password (default): username and password, sent to Trident's POST /auth/login. Trident gets the token.
• device: interactive login in the browser. The tool prints a URL and a code, you log in there (MFA and SSO work, the password never reaches the tool). Uses the public client undertow.
• client_credentials: client id and secret get tokens directly from the OIDC provider. No user, no password, no browser; meant for CI, provisioning and other automations.

device and client_credentials work with any OIDC provider, e.g. Keycloak. The tool finds the provider's endpoints through <issuer>/.well-known/openid-configuration; it stops with an error when this document cannot be read or, for device, has no device_authorization_endpoint.

Password:

undertow --url https://trident.example.org/tti -u alice -p '...' system login
undertow system user

system login stores URL, username and password in ~/.undertow/credentials; system logout removes them and the cached tokens.

Device:

export UNDERTOW_OIDC_ISSUER=https://id.example.org
undertow --url https://trident.example.org/tti --auth device system login

Later calls reuse and refresh the cached tokens. The provider needs a public client undertow with the OAuth 2.0 Device Authorization Grant enabled whose tokens carry Trident's audience (trident); eks-cli addon trident prepare creates it.

Client credentials:

export UNDERTOW_AUTH=client_credentials
export UNDERTOW_OIDC_ISSUER=https://id.example.org
export UNDERTOW_OIDC_CLIENT_ID=ci-client
export UNDERTOW_OIDC_CLIENT_SECRET=...

This needs a confidential client with service accounts enabled whose tokens carry Trident's audience (trident).

Use the same issuer as Trident, otherwise the token's issuer does not match. The user or service account needs Trident's roles for the routes it calls.

Commands

undertow --help and undertow <group> --help list everything. The main ones:

# files
undertow files list --search report --all
undertow files upload ./measurement.h5 --name "Measurement A"
undertow files sync --folder ./data
undertow files download measurement.h5 --to ./downloads/
undertow files inspect ./measurement.h5          # HDF5 metadata, needs undertow[hdf5]

# services and offerings
undertow services create --name "Measurement A" --type dataset --file-id 12
undertow offerings create --name "Measurement A" --service-id 5 --type compute --trust-list-id 2
undertow offerings publish 7
undertow offerings revoke 7

# dataspace
undertow dataspace list
undertow dataspace access did:op:...               # the server orders and downloads it
undertow dataspace compute start -d did:op:dataset -a did:op:algorithm --wait
undertow dataspace compute status <job id>
undertow dataspace results list --type compute
undertow dataspace results outputs 42
undertow dataspace results publish 42 --path outputs/model.json            # shows the draft
undertow dataspace results publish 42 --path outputs/model.json --yes      # publishes

# master data
undertow masterdata participants list
undertow masterdata trust-lists ensure --name "Partners" --participant-ids tenant-a,tenant-b
undertow masterdata docker-images ensure --name example/algorithm-base --tag 1.0 --checksum sha256:...

Provisioning

undertow system provision --config provisioning.yaml --dry-run
undertow system provision --config provisioning.yaml
undertow system sync --config provisioning.yaml      # only the folder sync

The file format is described in Trident's docs/PROVISIONING.md. undertow sends the master data of the file to POST /management/provisioning of every participant with a server entry (URL plus username/password or server.auth), then syncs its sync_folder. Values like ${tenant-a.server.password} are taken from a secrets file: --secrets-file, UNDERTOW_SECRETS_FILE, ~/.undertow/secrets.yaml, or .secrets.yaml / secrets.yaml next to the provisioning file or in the working directory. Environment variables work too, and ${NAME:-default} sets a default.

# secrets.yaml
tenant-a:
  server:
    password: "<password>"

Python

from undertow import TridentClient

client = TridentClient("https://trident.example.org/tti", "alice", "...")
res = client.management.upload_file("measurement.h5", name="Measurement A")
print(res.json()["id"])

for participant in client.management.list_all("participants"):
    print(participant["tag"])

Tokens from the OIDC provider: TridentClient(url, auth=undertow.auth.auth_from_env()).

Development

python3 -m venv .venv && . .venv/bin/activate
make install     # pip install -e ".[dev]"
make test

The tests use mocks only and need no server.

Release

The version is set in src/undertow/__init__.py (__version__).

make init        # build tools
make release     # checks PyPI and the git tree, runs the tests, builds, uploads, tags v<version>

License

MIT, see LICENSE.