Vortex CLI

Vortex CLI is a command line alternative to the Puakma Vortex IDE that simplifies the process of developing Puakma Applications on a Puakma Tornado Server using Visual Studio Code. It allows you to clone applications from the server to a local workspace, edit the files using Visual Studio Code, and automatically upload changes to the server as you work.

Vortex CLI also comes pre-packaged with the necessary Puakma .jar files for development.

Visual Studio Code and Extensions

While it is possible to use without it, this software has been purposefully designed for use with Visual Studio Code and the Project Manager For Java or the Extension Pack For Java extension. This software leverages Workspaces in Visual Studio Code and manages a vortex.code-workspace file within the workspace.

Installation

1. Install the tool using pip.

   pip install vortex-cli
   

2. It is recommended to set the workspace you would like to work out of via the VORTEX_HOME environment variable.

   On Unix:

   export VORTEX_HOME=/path/to/workspace
   

   Otherwise, Vortex CLI will use a default 'vortex-cli-workspace' directory inside your home directory.

3. Run vortex with the --init flag to create your workspace (If it doesn't already exist) and the necessary config files:

   vortex --init
   

4. Define the servers you will be working with in the servers.ini file inside the .config directory within your workspace. You can quickly access this using the code command to view your workspace in VSCode.

   vortex code
   

   In the servers.ini file, you can define as many servers as you need, each with their own unique name. For example:

   [DEFAULT] ; This section is optional and only useful if you have multiple definitions
   port = 80 ; Options provided under DEFAULT will be applied to all definitions if not provided
   soap_path = system/SOAPDesigner.pma
   default = server1 ; Useful when you have multiple definitions
   
   
   [server1] ; This can be called whatever you want and can be referenced using the '--server' flag
   host = example.com
   port = 8080 ; we can overwrite the DEFAULT value
   username = myuser ; Optional - Prompted at runtime if not provided
   password = mypassword ; Optional - Prompted at runtime if not provided
   ; Optional
   puakma_db_conn_id = 13 ; Optional - discovered from the server when omitted
   backend = soap ; 'soap' (default) or 'gateway' - see The Agent Gateway below
   gateway_path = vortex/gateway.pma ; only used when backend = gateway
   clone_with_resources = html,css,js ; resources with these extensions are always cloned - 'clone --get-resources' still clones ALL resources
   lib_path = ; optional extra jars to add to the classpath (the server's own jars are downloaded automatically - see 'vortex libs')
   workspace_folders = ~/dev/shared,notes ; extra folders to mount in the generated .code-workspace files. Relative paths resolve against the workspace root. Under [DEFAULT] they are added to every workspace; here they apply to this server's workspace (and the global one)
   java_home = /usr/lib/jvm/java-17-openjdk-amd64/ ; The local path to the JRE to use. Should be the same version running on your server
   java_environment_name = JavaSE-17 ; Java Execution Environment name https://docs.osgi.org/reference/eenames.html
   

Upgrading to 6.3

• vortex push only uploads what changed. On a backend = gateway server the push first reads the server's element hashes and skips every file that already matches - a full-app push that used to re-send (and journal) hundreds of identical files now sends the few that differ. --all restores the old push-everything behaviour; backend = soap still pushes everything.
• Uploads run concurrently - up to --jobs N elements at once (default 8; -j 1 is the old one-at-a-time push). A Java element's source and class still go in that order, and a failed source stops that element's class from being uploaded over it.
• push --name prefers an exact match. Previously --name dashboard also pushed a Dashboard element; now the case-insensitive match only applies when nothing matches exactly.

Upgrading to 6.0

6.0 is additive - the default behaviour is unchanged. Every server keeps using the SOAP designer path unless you opt it in.

• New optional backend: the agent gateway. Set backend = gateway on a server definition to route supported operations through the vortex/gateway Puakma application instead of SOAPDesigner. The default is backend = soap, which never contacts the gateway at all - so servers without it installed are unaffected. See The Agent Gateway.
• New commands: status, agenda, push, pull, render and undo. agenda, pull, render and undo require backend = gateway; status and push fall back to the existing paths.
• puakma_db_conn_id is now optional. When omitted it is discovered from the server on both backends. Existing configs that set it explicitly keep working.
• vortex list gains Version and Last Modified columns when run against a gateway server.

Upgrading to 5.0

5.0 changes some defaults you may rely on:

• resource_ext_only is replaced by clone_with_resources - and the meaning flipped. Previously the extensions restricted what --get-resources cloned. Now resources with the listed extensions are always cloned, and --get-resources clones every resource unfiltered. Rename the key in servers.ini (vortex warns while the old key is present).
• vortex watch now watches every cloned app across all servers and uploads each change to the server it was cloned from. Use --server for the old single-server behaviour, and mark production definitions protected = true so they are never watched by accident.
• vortex compile now uses the Eclipse compiler (ecj) - the same compiler the VS Code Java extension uses - downloaded once per workspace. --javac PATH forces javac. compile --upload is now a full refresh (uploads every compiled class, not just changed ones).
• find, grep and vortex list --local now search all cloned apps unless --server is given, and ID-taking commands infer their server from local clones (see below).

Usage

For a full list of commands see --help.

Command Overview

• code: Open the workspace in Visual Studio Code (-s <server> opens that server's own workspace with exactly its jars on the Java classpath).
• use: Set the default server so you don't need to pass --server on every command. e.g. vortex use production
• list (or ls): List Puakma Applications on the server or cloned locally. (ls is an alias for vortex list --local)
• clone: Clone Puakma Applications and their design objects into the workspace. Apps can be referenced by ID (vortex clone 13), by TemplateName (vortex clone bettrackr_app), or by group/name (vortex clone bettrackr/app). A bare application group clones every active, non-inherited application in it (vortex clone BetTrackr; add --all/-a for the disabled and inherited ones too) - all optionally server-qualified (dev:13, dev:bettrackr/app, dev:BetTrackr). --reclone re-clones what is already cloned: every server's clones, or one server's with --server. See Cloning a whole group.
• watch: Watch the workspace for changes to Design Objects and automatically upload them to the server each app was cloned from. Watches all servers at once unless --server is given.
• clean: Delete the locally cloned Puakma Application directories in the workspace. Takes optional APP_IDs to clean just those clones (all of the server's, if none are given). Deletes immediately and needs no network - there is no undo. With --check it first asks the server for its design element hashes and refuses to delete a clone holding local changes the server doesn't have, naming each file; that needs a reachable backend=gateway server, and a clone it can't verify is refused rather than silently deleted.
• config: View and manage configuration. --check-gateway reports the agent gateway negotiation and which gateway roles your identity holds.
• log: View the server log.
• status: Show the server status - via the agent gateway when available, otherwise the raw console status output.
• agenda: List every scheduled action with its decoded schedule, last and next run, and whether it is overdue (read-only, gateway only).
• push: Upload local design files (source, compiled classes, pages, resources) to the server without a watch session - journaled and deploy-confirmed when routed via the gateway. On a gateway server only files whose content differs from the server's are uploaded (compared by the server's element hashes; --all pushes everything), and elements upload concurrently (--jobs N, default 8) while each Java element's source always lands before its class. --name matches exactly first, case-insensitively only when nothing matches exactly.
• pull: Refresh cloned applications in place via the gateway's incremental sync - only elements changed since the last sync are downloaded. Refuses to overwrite locally modified files without --force.
• render: Render a PAGE server-side as your identity and print it (or --out FILE) - see a change without a browser login (gateway only).
• undo: The server-side undo journal. No arguments lists journal entries; with an entry id it dry-runs a restore, and --confirm writes the journaled version back (gateway only).
• find: Find Design Objects of cloned applications by name.
• grep: Search the contents of cloned Design Objects using a Regular Expression.
• new: Create new Design Objects, Applications, or Keywords. Use --update <ID> to update instead. Run without flags to launch an interactive wizard.
• keyword (or kw): Read or update an application's Keyword values. vortex keyword 9 lists them, -n <Name> filters to one, and -n <Name> --values ... updates it, printing the prior value beside the new one before asking to continue. Secret-looking names are redacted unless --reveal is given - see Keyword values and secrets.
• copy: Copy a Design Object from one application to another.
• delete: Delete Design Objects by ID.
• db: Interact with Database Connections. Accepts --server/-s like other commands for one-off queries against another server.
• schema: Manage the Puakma data dictionary (PMATABLE/ATTRIBUTE): record table/column definitions and print the DDL to run by hand. Never executes DDL.
• compile (or build): Compile an application's Java Design Objects into zbin/ using the Eclipse compiler (ecj).
• libs: Show or refresh the per-server Java library cache (each server's puakma.jar and shared libraries, downloaded from the server itself).
• docs: Open the Tornado Server Blackbook.
• execute: Execute a command on the server.

Keyword values and secrets

Keywords are an application's live configuration, and in practice they hold credentials in cleartext - API keys, passwords, vendor secrets, signing keys. vortex keyword therefore redacts by default, following the same precedent as vortex config --show-server:

vortex keyword 9                      # list every Keyword (secrets redacted)
vortex keyword 9 -n AppVersion        # just this one
vortex keyword 9 -n AppVersion --values 3.7.0    # update it
vortex keyword 9 --reveal             # print secret values in full

• A Keyword is treated as secret when its name contains a marker such as password, passwd, passphrase, secret, credential, signature, apikey, privatekey, keystore, webhook, connectionstring or mnemonic, or when any word of the name is one of key, keys, pwd, token, salt, hash, private, auth, bearer, cert, pem, jwt, dsn, otp, pin, seed or sig. Names are split on camelCase/snake_case/kebab-case, so AccessKeyId and API_KEY are redacted while Monkey and Concert are not. The match is on the name only - a value-based heuristic would leak the thing it is trying to hide on every near miss. Classification is deliberately biased towards over-redaction: a false positive costs one --reveal, a false negative prints a live credential.
• Redacted values print as <redacted>, one per value, so the value count stays visible.
• --reveal is the explicit opt-in. Without it, neither a listing nor an update's before/after diff can put a live credential into your terminal scrollback, your shell history or a CI log.

An update always prints the prior value beside the new one and waits for [Y/y] on stdin before writing; on a protected = true server the server name must also be typed back (--yes does not bypass that). The write uses the gateway's keyword upsert, so it never inserts a duplicate KEYWORD row.

Reading does not require the application to be cloned locally - diagnosing a bad configuration value should not depend on having a working copy.

Cloning a whole group

vortex clone takes an ID, a TemplateName, an application group, or group/name:

vortex clone 13                 # by ID
vortex clone bettrackr_app      # by TemplateName
vortex clone bettrackr/app      # by group/name
vortex clone BetTrackr          # every active application in the BetTrackr group
vortex clone -a BetTrackr       # ... including its disabled and inherited ones
vortex clone dev:BetTrackr      # ... on the 'dev' server
vortex clone --group BetTrackr  # explicitly a group, never a TemplateName

A group clone skips disabled and inherited applications (the ones vortex list hides by default) unless --all/-a is given. An application you name outright - by ID, TemplateName or group/name - is always cloned, whatever its state.

Groups are matched the way vortex list --group matches them - a case-insensitive substring - except that an exact (case-insensitive) group name always wins, so cloning BetTrackr never drags in BetTrackrLegacy. If a partial name still spans several groups, vortex stops and lists them rather than cloning the lot.

A bare word is looked up as both a TemplateName and a group. In the rare case that it is genuinely both, vortex refuses to guess: use --group NAME for the group, or the ID / group/name for the single application. Every flag (--reclone, --all, --get-resources, --open-urls, --timeout, --server) applies to group clones as it does to single apps.

Working with Multiple Servers

Each section in servers.ini defines a server (hosts must be unique across sections). Cloned apps remember which server they came from, so you can work on apps from several servers at the same time:

• vortex watch watches every cloned app and uploads each change to the server it was cloned from. One terminal, all servers. Log lines are prefixed with the server name (e.g. [dev] Upload DATA of ...). Use --server to watch a single server only.
• Commands that take IDs (clone, export, compile, delete, copy) work out the server on their own: IDs can be qualified as SERVER:ID (e.g. vortex export dev:123), and unqualified IDs are resolved against your locally cloned apps. If an ID exists on more than one server, vortex stops and asks rather than guessing.
• find, grep and vortex list --local search across all cloned apps unless --server is given.
• watch, clone and clean take a workspace-wide lock: only one watch at a time, and cloning or cleaning is refused while a watch is running (stop the watch first - a clone under a running watch adds directories the watcher doesn't know about). Finer-grained commands that alter design elements (delete, copy, new, compile --upload) lock per-application: they are refused for apps a watch is watching and run concurrently otherwise. push and pull lock the same way - per application, never workspace-wide - so they can run against one app while a watch holds others.
• In VS Code, app folders are listed in per-server blocks (dev: group/app, ...) with the server's jars on the Java classpath (see them in the Java Projects view). vscode-java's classpath settings are window-scoped - folder-level settings are ignored - so the global workspace unions every server's jars (version clashes between servers are possible there). For a guaranteed-correct classpath, open a single server's own workspace with vortex code -s <server> (generated as .vscode/<server>.code-workspace, containing only that server's apps and exactly its jars).

vortex use <server> still sets the default server for commands that have no IDs to infer from (list, log, db, execute, new):

vortex use production
vortex list

Credentials

username/password can be left out of servers.ini. Each server's credentials resolve in this order and are only requested when a command actually connects to that server:

1. The system keyring - store with vortex config --set-password -s <server> (requires pip install keyring)
2. Per-server environment variables VORTEX_USERNAME_<SERVER> / VORTEX_PASSWORD_<SERVER> (e.g. VORTEX_PASSWORD_DEV)
3. VORTEX_USERNAME / VORTEX_PASSWORD
4. An interactive prompt naming the server

Protected Servers

Set protected = true on a server definition (e.g. production) to make it hard to change by accident:

• Write operations (delete, copy, new, import, db --update, execute, schema changes, compile --upload) require the server name to be typed back to continue. This is deliberately not bypassed by --yes.
• vortex watch skips protected servers unless --include-protected is given, so saving a file can never hot-deploy to production by accident.

The Agent Gateway

The agent gateway is a JSON API served by the Puakma server itself - a companion Puakma application (vortex/gateway) that you deploy to a server. Where SOAPDesigner is a transport, the gateway is a transport plus the guarantees only server-side code can enforce:

• Role-gated operations - every endpoint requires one declared application role, checked on the server. Roles are rows in that server's copy of the app, so a grant on dev confers nothing on prod.
• An undo journal - destructive design writes snapshot the element first (bytes, metadata, design params). vortex undo lists those snapshots and restores any of them; deleted elements are recreatable.
• Dry-run by default - undo, element deletes, single-statement DML and whole-server refresh do nothing without an explicit confirmation.
• Deploy confirmation - writes reply with the sizes and hashes of what is really on the server now.
• Database guardrails - the Puakma system database is unreachable through it, and DDL is never executed, only generated as text.

Opt in per server:

[dev]
host = dev.example.com
backend = gateway            ; default is 'soap', which never contacts the gateway
; gateway_path = vortex/gateway.pma   (this is the default)

With backend = gateway, watch/push uploads, delete, log, status, db, execute and the journal commands route through it, and agenda, pull, render and undo become available. Everything the gateway does not offer goes to the webdesign vortex API (the JSON action served by system/webdesign) - see SOAP-free on backend = gateway. A backend = gateway server never talks to SOAPDesigner at all. Check what you are talking to and what you may do:

vortex config --check-gateway -s dev

There is no fallback in either direction, by design. A gateway refusal (for example, your identity lacks the required role) is reported as an error - vortex never quietly retries the same operation over SOAP, because that would let anyone with SOAP access bypass every role, journal and guardrail above. Likewise backend = soap never contacts the gateway. The one deliberate exception is the gateway application's own deployments, which never go through the gateway (webdesign on backend = gateway, SOAP on backend = soap) so that a broken gateway deploy never needs the gateway to fix itself.

The gateway application is not bundled with this CLI - deploy it to a server with vortex export / vortex import, then run its Setup scheduled action. It ships its own API.md, README.md and DECISIONS.md as DOCUMENTATION design elements, so once installed the running server documents its own endpoints, roles and setup steps.

Note: the gateway's roles are only a real boundary for an identity whose sole route to the server is the gateway. Any identity that can reach system/webdesign or deploy code can grant itself any role. Keep webdesign access for operators; agent identities should not have it.

SOAP-free on backend = gateway

SOAPDesigner is legacy. On a backend = gateway server the commands the gateway does not cover use the JSON API of system/webdesign's vortex action instead - with the same credentials, and the webdesign app's own ACL. SOAP is used only when a server is explicitly backend = soap. The data dictionary (schema, db --list/--schema) is the exception that proves the rule: the gateway's dictionary endpoint runs that same webdesign vortex action server-side behind GatewayDBRead/GatewayDBWrite, so an agent identity with no webdesign access gets a clean role answer rather than a login page, and a change to vortex.java needs no change to the gateway.

Where the webdesign API behaves differently from SOAP, the CLI compensates so the commands behave as they did. The differences that remain visible:

• Design element writes are whole-row. The CLI reads the row back first and re-sends what it does not model (the other blob, Options), so a DATA-only upload never wipes source. One consequence: every write is one extra GET.
• Renaming with new object --update --name does not rewrite references. SOAP's updateDesignObject also updated design params in the app that referred to the old name; the webdesign route does not. Fix OpenAction/ParentPage-style params by hand after a rename.
• schema cannot record --default, --position or the column half of --ref. The webdesign column write has no DefaultValue/Position/RefColumn fields (SOAP's savePuakmaAttribute2 had them). --default/--position are refused with a message; --ref TABLE.COLUMN records the table and warns. Updates leave existing values untouched. Extend POST/PUT .../column in webdesign's vortex.java to lift this.
• Every write flushes the application's design cache, where SOAP flushed one element.
• No per-application Developer role check - the webdesign app's ACL is the boundary. An identity that can reach system/webdesign can edit every application through it.
• db --list orders by table name (SOAP's SELECT DISTINCT had no order); --schema output is identical.
• Database-name resolution is app-scoped. schema/db --list find the connection whose dictionary holds the tables (as SOAP did by pmatable count), asking locally cloned applications first and scanning the server inventory only if none of them has it.
• import remains SOAP on every backend until webdesign gains a PMX import route (SaveImportPMX is a multipart UI form, not an API).

Interactive Wizards

Run vortex new object or vortex new app without any flags to launch a step-by-step wizard:

vortex new object
# → pick app from locally cloned list
# → pick design type
# → enter name
# → enter content type (resource/documentation only)
# → enter parent page, open action, save action (page only)
# → preview and confirm

vortex new app
# → enter name
# → pick from existing groups or enter a new one
# → enter description (optional)
# → preview and confirm

Skipping Confirmation Prompts

Pass --yes (or -y) to skip Y/y confirmation prompts, useful for scripting:

vortex --yes delete 123 456
vortex -y new object --name MyAction --app-id 10 --type action

Note: --yes never bypasses the typed confirmation for servers marked protected = true.

Compiling Java Design Objects

vortex compile (alias build) compiles an application's Java Design Objects into zbin/ with the Eclipse compiler (ecj) - the same compiler the VS Code Java extension uses, downloaded once per workspace into config/.tools/. It runs via the java from java_home in the server config, then $JAVA_HOME, then PATH; --release is derived from the application's Java class version. ecj matters because Tornado loads each class from its own design element: '$' classes (Foo$1.class) are never uploaded to the server, and unlike javac, ecj compiles a switch over an enum into the class itself rather than a synthetic Foo$1.class. Classes that still produce $ files (genuine anonymous/inner classes) are reported and excluded from upload - they would throw NoClassDefFoundError on the server. If ecj can't be downloaded (offline), javac is used with a warning; --javac PATH forces javac explicitly.

vortex compile              # compile all locally cloned apps for the server
vortex compile 13           # compile one app
vortex compile 13 --upload  # also upload ALL compiled classes (full server refresh)

While vortex watch is running, --upload isn't needed: saving a Java source uploads the SOURCE, and the VS Code Java extension's incremental build (which is the Eclipse compiler, ecj - the same compiler vortex compile downloads) writes the class files into zbin/, where watch picks them up and uploads the DATA - always, even when the bytes are unchanged, so a source change is always paired with its class on the server. '$' nested-class files are never uploaded - a class compiled alongside $ siblings is refused with an error since it would fail on the server. Use vortex compile for full-application rebuilds without the IDE.

Server-Provided Java Libraries

Compilation and IDE IntelliSense need the Puakma framework jar and the server's shared libraries on the classpath. Instead of maintaining local copies, vortex downloads them from each server's webdesign application (the vortex API's systemjar and libraries endpoints) and caches them per server under <workspace>/<host>/.lib/:

• The cache is filled automatically the first time it's needed - on clone, watch and compile - and never re-downloaded unless you ask.
• vortex libs shows what's cached for each server; vortex libs --refresh re-downloads (add -s <server> for one server), e.g. after a server upgrade.
• javac (via vortex compile) and each app folder's VS Code Java settings use the app's own server's cached jars, so identical class names on different servers/versions never cross-contaminate.
• Servers whose webdesign app doesn't provide the vortex API yet fall back to the puakma.jar bundled with vortex-cli plus any lib_path entries, with a warning.
• vortex clean keeps each host's .lib cache so the jars don't need re-downloading; pass --include-libs to remove them as well.

Managing the Data Dictionary

vortex schema records design-time table and column definitions in the Puakma data dictionary (the PMATABLE/ATTRIBUTE tables, which also back vortex db --schema) and prints the matching DDL for you to run by hand. It never executes DDL, and deletes only remove dictionary rows - never real tables or columns.

vortex schema mydb --add-table invoice --description "Customer invoices"
vortex schema mydb --add-column invoice invoice_id --type BIGINT --pk --auto-increment
vortex schema mydb --add-column invoice total --type NUMERIC --size 10,2 --not-null
vortex schema mydb --add-column invoice customer_id --type BIGINT --ref customer.customer_id
vortex schema mydb --ddl invoice    # print CREATE TABLE from the dictionary

On a backend = gateway server the dictionary is reached through webdesign's vortex API via the gateway's dictionary endpoint (GatewayDBRead to read, GatewayDBWrite to change). That API cannot record --default or --position (refused) nor the column half of --ref (warned) - see SOAP-free on backend = gateway.