ewms-pilot v1
An Event-Task Pilot for EWMS
The EWMS Pilot is a non-user-facing wrapper for task container instances in the Event Workflow Management System (EWMS), running on an HTCondor Execution Point (EP). The pilot:
- Triggers task instances for each inbound event.
- Interfaces with EWMS events as input/output files.
- Isolates task containers from one another.
- Provides fault tolerance for failed tasks, CPUs, etc.
The following outlines what users need to know to operate within EWMS.
Overview
The Pilot is designed to be invisible to users. However, some key details are necessary for running a task container:
Task Container Overview
A task container is created for each inbound event, it is defined by its image, arguments, and environment variables. See the WMS docs for information on setting these within EWMS.
Event I/O
An input event is provided to the task container as a file. The task container creates an output event by writing to a predetermined location.
The pilot provides the filepaths to the input and output files in two ways:
- By replacing the placeholder strings,
{{INFILE}}and{{OUTFILE}}, in the container's arguments at runtime. - By setting the task container's environment variables:
EWMS_TASK_INFILEandEWMS_TASK_OUTFILE.
The files' extensions are configured by the pilot's environment variables, EWMS_PILOT_INFILE_EXT and EWMS_PILOT_OUTFILE_EXT: by default, these are .in and .out, respectively.
No other event or message handling is required by the task container.
The Init Container
An init container is an optional, user-supplied image used to set up the environment, wait for conditions, or perform other preparatory actions before running task containers. It is configured using the EWMS_PILOT_INIT_IMAGE, EWMS_PILOT_INIT_ARGS, and EWMS_PILOT_INIT_ENV_JSON environment variables.
File I/O
Task containers (and init containers) can interact with external files in two ways:
Inter-Task Files
To transfer files between task containers, a shared directory is available to all task containers and the init container.
The pilot provides the filepath to the "data hub" in two ways:
- By replacing the placeholder string,
{{DATA_HUB}}, in the container's arguments at runtime. - By setting the task container's environment variable:
EWMS_TASK_DATA_HUB_DIR.
Note:
- The data hub directory is writable, but there is no protection against race conditions for parallelized tasks.
External Files
Externally-mounted directories are supported in EWMS. See the WMS documentation for more details.
EWMS Glossary Applied to the Pilot
Workflow
Does not exist within the Pilot. Compare to WMS.
Message Queue
The message queue is abstracted from the task container and can be ignored. Compare to WMS.
Event
An event is an object transferred via event I/O. Compare to WMS.
Task
In the context of the Pilot, the task is the runtime instance of the task image (a task container) applied to an inbound event, possibly producing outbound events. Compare to WMS.
Task Container
The task container is an instance of a task image and is nearly synonymous with task.
Task Directive
Does not exist within the Pilot. Compare to WMS.
Taskforce
Does not exist within the Pilot. Compare to WMS.
1 maintainer
Metadata
1 maintainer
Release files for ewms-pilot 1.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| ewms_pilot-1.0.5.tar.gz | 28.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| ewms_pilot-1.0.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 60.3 kB
Release files / ewms_pilot-1.0.5.tar.gz
| Download URL | ewms_pilot-1.0.5.tar.gz |
|---|---|
| Size | 28.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6ca2119e6aa3b8312d411c554a1d71669560ded36302a3376fb12e0119337c3f
|
|
BLAKE2b-256 checksum How to use checksums |
13afee8032b75cddbd83be590bba11957f4aeeb741104d1ebe91e6134083edd6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.8
|
Release files / ewms_pilot-1.0.5-py3-none-any.whl
| Download URL | ewms_pilot-1.0.5-py3-none-any.whl |
|---|---|
| Size | 31.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b17ece81a42af19d648a79b17a9a4f020368c04704fc64452c544c21316b0be9
|
|
BLAKE2b-256 checksum How to use checksums |
497b5f6f7e7f00a162fa99fdfba770c22d0b22fa8ee8780e63db2cb5d31663f0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.8
|
