ztl 0.1.5

A thin library relying on zmq to dispatch tasks

This project contains the ZTL library enabling light-weight and widely compatible remote task execution using ZeroMQ.

System requirements:

• ZTL is compatible with all major operating systems running Python and is independent of any specific hardware.
• ZTL has been tested on Ubuntu versions 16.04, 18.04, 20.04, 22.04, 24.04.
• ZTL has the following dependencies:
  • pyzmq
  • oyaml
  • pytest, pytest-xprocess (for testing only)

Quick usage (modern Ubuntu)

If you know what you are doing and operate a modern Ubuntu using Python 3, follow these simple steps for installing ZTL in a virtual environment. You can then skip the detailled Installation Guide and try the Demo directly.

> sudo apt install python3-pip python3-venv
> python3 -m venv ~/ztl
> source ~/ztl/bin/activate
> pip install ztl

Detailled installation guide

You can easily install and use ZTL in a Python virtual environment in a two-step process. First, we need to setup the virtual environment and then install the library depending on your preferences. The process described covers various Ubuntu versions but the library can also be installed and used in Windows using using slightly different commands.

Preparing the virtual environment

To create such an environment in your home directory, follow the below steps. You can install into any existing virtual environment or change the installation folder from ~/ztl to any other folder you prefer.

For modern Ubuntu-based systems using Python 3, the first step is to install the necessary packages and then to create and activate a virtual environment. Package installation will need administrator rights and might fail. Installation on other operating systems might also vary, but you should make sure to have pip and venv modules for python ready.

> sudo apt install python3-pip python3-venv
> python3 -m venv ~/ztl

For older Ubuntu systems using Python version 2, please use the following steps instead:

> sudo apt install python-pip python-virtualenv
> virtualenv ~/ztl

No matter which python version, next you need to load this environment. You have to repeat this step for every terminal you want to use the library in:

> source ~/ztl/bin/activate

Installing the library

There are multiple possibilities to install the library on your system. All are valid alternatives, however, some are restricted to specific Python versions.

Option 1 (pip)

On modern systems with Python 3, you can simply use pip for installation. Add the option --upgrade to update to the latest version.

> pip install ztl

Option 2 (zip file)

Alternatively, you can download and unpack the source code, independent of your Python version. Please check the .zip file for any subfolders containing the actual files (e.g. when you download from the gitlab repository, it will create a subfolder called ztl-main).

> unzip /path/to/ztl.zip -d ~/ztl-src
> pip install ~/ztl-src

Option 3 (repository)

You can also clone the repository directly and install the latest sources, independent of your Python version. If you replace main with a branch or tag, you can install specific versions.

> pip install git+https://gitlab.com/robothouse/rh-user/ztl@main

Demo

To test the correct installation of ZTL, you can use the following steps. Please remember to activate the virtual environment for each terminal.

> source ~/ztl/bin/activate

Simple client-server communication

First, spawn a server listening on port 12345 and the scope /test:

> ztl_task_server -p 12345 -s /test
INFO:remote-task:Task Server listening at 'tcp://*:12345'
INFO:remote-task:Registering controller for scope '/test'.

The output confirms that the server component is listening at the specified port and spawns a controller to handle tasks on scope /test. Then, call the server at the same port (on the local machine) under the same scope /test using a client (in a different terminal) to see how it replies:

> ztl_task_client -r localhost -p 12345 -s /test some-handler:executing-component:goal-state
Connecting to host 'localhost:12345' at scope '/test'...
INFO:remote-task:Remote task interface initialised at 'tcp://localhost:12345'.
Triggering task with request 'c29tZS1oYW5kbGVywqxleGVjdXRpbmctY29tcG9uZW50wqxnb2FsLXN0YXRl'...
Initialised task with ID '1' for 5s. Reply is 'Initiated task '1' with request: c29tZS1oYW5kbGVywqxleGVjdXRpbmctY29tcG9uZW50wqxnb2FsLXN0YXRl'.
Task with ID '1' finished in state 'COMPLETED' while waiting. Result is 'finished'.

The output shows successful connection with the server, the creation of a remote task as an interface to the server, whic then gets triggered. The reply that the client receives indicates that the server has successfully initialised a task and given it the ID 1. After completion of the task, the client receives a completion message with the result "finished". If we now look back at the server side, we find that the server also provided some insights about the communication.

Initialising Task ID '1' (c29tZS1oYW5kbGVywqxleGVjdXRpbmctY29tcG9uZW50wqxnb2FsLXN0YXRl)...
handler: some-handler
component: executing-component
goal: goal-state
Status Task ID '1' (waiting poll): ACCEPTED.
Status Task ID '1' (waiting poll): ACCEPTED.
...
Status Task ID '1' (waiting poll): COMPLETED.

We find that the client first initialises a task (with the specification of handler, component, and goal) and then polls the server periodically about the status of the task it has provided.

Scripting example

To test the scripting interface, you can use the configurations provided in the example folder (Note: these are not installed using pip but are part of the source code package). First, we need to start a task server, for which one of the example servers is sufficient as it provides all the necessary communication with the scripting engine. Please remember to activate the virtual environment for each terminal.

> ztl_task_server -p 7779 -s /test

Next, download the configuration files if you do not have them yet.

> git clone https://gitlab.com/robothouse/rh-user/ztl ~/ztl-src

Run the sample scripts:

> cd ~/ztl-src/src/ztl/example/
> ztl_run_script -s sample_script_short.yaml -c sample_conf.yaml
INFO:script-exec:Initialising remote task interface 'testhandler'...
INFO:remote-task:Remote task interface initialised at 'tcp://localhost:7779'.

----------------------------
ABOUT TO EXECUTE SCENE 'scene'
STEP: first step
        testhandler [print]: -> test me
STEP: second step
        testhandler [print]: -> another test
PRESS <ENTER> TO CONFIRM or ANY OTHER KEY TO SKIP

You can now trigger the example scene by pressing ENTER, which will execute a single scene consisting of two steps in sequence, both calling the testhandler we started above. You can also skip the scene to finish the script immediately. Again, the server will output the triggering, acceptance, and completion of both calls. You can modify the script to call multiple handlers in each step but you have to configure them in the file sample_conf.yaml and start another ztl_task_server to match this configuration.

Instructions for use

To implement your own server in Python, you can use the following as an example. This code, however, will reject any task and never execute the status() or abort() functions since the task is never initialised correctly. For a slightly longer working example including an executable task, refer to task_server.py in the examples.

from ztl.core.task import TaskController
from ztl.core.protocol import State
from ztl.core.server import TaskServer

class NoneController(TaskController):

    def init(self, request):
        return -1, "Not implemented"


    def status(self, mid, request):
        return State.FAILED, "Not implemented"


    def abort(self, mid, request):
        return State.REJECTED, "Not implemented"

server = TaskServer(12345)
server.register("/test", NoneController())
server.listen()

A client to trigger the above server can be implemented as follows:

from ztl.core.client import RemoteTask
from ztl.core.protocol import State, Task

task = RemoteTask("localhost", 12545, "/test")
request = Task.encode("some-handler", "executing-component", "goal-state")
task_id, reply = task.trigger(request)

Architecture overview

The basic communication principle is as follows:

Thereby, each task has the following lifecycle:

An example communication could look like this:

Request:

Reply: