wrapyfi 0.5.0

Wrapyfi is a wrapper for simplifying Middleware communication

Wrapyfi is a middleware communication wrapper for transmitting data across nodes, without altering the operation pipeline of your Python scripts. Wrapyfi introduces a number of helper functions to make middleware integration possible without the need to learn an entire framework, just to parallelize your processes on multiple machines. Wrapyfi supports YARP, ROS, ROS 2, ZeroMQ, Websocket, Zenoh and MQTT.

Attribution

Please refer to the following paper when citing Wrapyfi in academic work:

@inproceedings{abawi2024wrapyfi,
  title = {Wrapyfi: A Python Wrapper for Integrating Robots, Sensors, and Applications across Multiple Middleware},
  author = {Abawi, Fares and Allgeuer, Philipp and Fu, Di and Wermter, Stefan},
  booktitle = {Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI '24)},
  year = {2024},
  organization = {ACM},
  isbn = {79-8-4007-0322-5},
  doi = {10.1145/3610977.3637471},
  url = {https://github.com/fabawi/wrapyfi}
}

Getting Started

Before using Wrapyfi, YARP, ROS, ZeroMQ, Websocket, Zenoh, or MQTT must be installed.

• YARP: Follow the YARP installation guide. The yarpserver server must be running before running any YARP-based scripts. Note that the iCub package is not needed for Wrapyfi to work and does not have to be installed if you do not intend to use the iCub robot.

• ROS: For installing ROS, follow the ROS installation guide [Ubuntu][Windows]. We recommend installing ROS on Conda using the RoboStack environment. The roscore server must be running before running any ROS-based scripts. Additionally, the Wrapyfi ROS interfaces must be built to support messages needed for audio transmission

• ROS 2: For installing ROS 2, follow the ROS 2 installation guide [Ubuntu][Windows]. We recommend installing ROS 2 on Conda using the RoboStack environment. Additionally, the Wrapyfi ROS 2 interfaces must be built to support messages and services needed for audio transmission and the REQ/REP pattern

• ZeroMQ: ZeroMQ can be installed using pip: pip install pyzmq. The XPUB/XSUB and XREQ/XREP patterns followed in our ZeroMQ implementation requires a proxy broker. A broker is spawned by default as a daemon process. To avoid automatic spawning, pass the argument start_proxy_broker=False to the method register decorator. A standalone broker can be found here

• Websocket: Websocket can be installed using pip: pip install python-socketio. The PUB/SUB pattern followed in our Websocket implementation requires a socket server. We recommend setting the server to run using Flask-SocketIO which can be installed with pip install flask-socketio. Note that the server must be running and also scripted to forward messages to the listening from the publishing client as demonstrated in the example found here

• Zenoh: Zenoh can be installed using pip: pip install eclipse-zenoh. It is recommended to use the WRAPYFI_ZENOH_MODE environment variable to set the mode to peer (default) for running in peer-to-peer mode. The PUB/SUB pattern followed in our Zenoh implementation requires a router. To install the Zenoh router, follow the instructions found here. The zenohd router must be running before executing any Zenoh-based Wrapyfi scripts with the environment variable WRAPYFI_ZENOH_MODE=client. NOTE: The zenohd --rest-http-port 8082 command must be executed with an arbitrary (non-conflicting) port to avoid collision with other services occupying the default port (8000).

• MQTT: MQTT can be installed using pip: pip install paho-mqtt. The PUB/SUB pattern followed in our MQTT implementation requires a broker. The default broker used by Wrapyfi broker.emqx.io. However, this broker is not recommended for production use or for transmitting video/audio as it is a public online broker and requires an internet connection (not secure and suffers high latency). We recommend setting up a local broker using Mosquitto. A Dockerized version can be found here. The broker must be running, and the WRAPYFI_MQTT_BROKER_ADDRESS as well as WRAPYFI_MQTT_BROKER_PORT environment variables must be set to the broker's address and port, respectively. When setting up a local broker with a username and password, they can be passed through the Wrapyfi method decorator as follows:

@MiddlewareCommunicator.register("NativeObject", "mqtt",
                                 "HelloWorld", 
                                 "/hello/my_message", 
                                 carrier="", should_wait=True,
                                 mqtt_kwargs=dict(username="username", password="password"))
def send_message(self):
    ...

Compatibility

• Operating System

  • Ubuntu >= 18.04 (Not tested with earlier versions of Ubuntu or other Linux distributions)
  • Windows >= 10 [beta support]:
    • Multiprocessing is disabled. ZeroMQ brokers spawn as threads only
    • Not tested with YARP and ROS 2
    • ROS only tested within mamba/micromamba environment installed using RoboStack
    • ROS and ROS 2 interfaces not tested
    • Installation instructions across Wrapyfi guides and tutorials are not guaranteed to be compatible with Windows 11
  • MacOS 10.14 Mojave

• Python >= 3.6

• OpenCV >= 4.2

• NumPy >= 1.19

• YARP >= v3.3.2

• ROS Noetic Ninjemys

• ROS 2 Humble Hawksbill | Galactic Geochelone | Foxy Fitzroy

• PyZMQ 16.0, 17.1 and 19.0

• Python-SocketIO >= 5.0.4

• Eclipse-Zenoh >= 1.0.0

• Paho-MQTT >= 2.0 (Hard-coded to v2 in Wrapyfi and not compatible with v1)

Installation

You can install Wrapyfi with pip or from source.

Pip

To install all the necessary components for the majority of common uses of Wrapyfi (e.g., NativeObject, Image, Audio, etc.) using pip, this process installs both Wrapyfi and its dependencies, like NumPy and OpenCV (opencv-contrib-python, opencv-headless, and opencv-python are supported), that are essential for various workloads, along with ZeroMQ being the default middleware. This option is the best for users running Wrapyfi out of the box in a newly created environment (without any middleware installed beforehand), installing numpy, opencv-contrib-python, and pyzmq:

pip install wrapyfi[all]

Note that most plugins require additional dependencies and should be installed separately.

or when installing Wrapyfi on a server (headless) including numpy, opencv-python-headless, and pyzmq:

pip install wrapyfi[headless]

Other middleware such as ROS are environment-specific and require dependencies that cannot be installed using pip. Wrapyfi could and should be used within such environments with minimal requirements to avoid conflicts with existing NumPy and OpenCV packages:

pip install wrapyfi

Source (Pip)

Clone this repository:

git clone --recursive https://github.com/fabawi/wrapyfi.git
cd wrapyfi

You can choose to install minimal dependencies including numpy, opencv-contrib-python, and pyzmq, for running a basic Wrapyfi script:

pip install .[all]

or when installing Wrapyfi on a server (headless) including numpy, opencv-python-headless, and pyzmq:

pip install .[headless]

or when installing Wrapyfi to work with websockets (headless) including numpy, opencv-python-headless, and python-socketio:

pip install .[headless_websockets]

or when installing Wrapyfi to work with Zenoh (headless) including numpy, opencv-python-headless, and eclipse-zenoh:

pip install .[headless_zenoh]

or when installing Wrapyfi to work with MQTT (headless) including numpy, opencv-python-headless, and paho-mqtt:

pip install .[headless_mqtt]

or install Wrapyfi without NumPy, OpenCV, ZeroMQ, Websocket, Zenoh, and MQTT:

pip install .

Docker

Wrapyfi Docker images can be pulled/installed directly from the modularml/wrapyfi repository on the Docker Hub. Dockerfiles for all supported environments can be built as well by following the Wrapyfi Docker instructions.

Usage

Wrapyfi supports two patterns of communication:

• Publisher-Subscriber (PUB/SUB): A publisher sends data to a subscriber accepting arguments and executing methods on the publisher's end. e.g., with YARP

Without Wrapyfi

With Wrapyfi

# Just your usual Python class class HelloWorld(object): def send_message(self): msg = input("Type your message: ") obj = {"message": msg} return obj, hello_world = HelloWorld() while True: my_message, = hello_world.send_message() print(my_message)

from wrapyfi.connect.wrapper import MiddlewareCommunicator class HelloWorld(MiddlewareCommunicator): @MiddlewareCommunicator.register("NativeObject", "yarp", "HelloWorld", "/hello/my_message", carrier="", should_wait=True) def send_message(self): msg = input("Type your message: ") obj = {"message": msg} return obj, hello_world = HelloWorld() LISTEN = True mode = "listen" if LISTEN else "publish" hello_world.activate_communication(hello_world.send_message, mode=mode) while True: my_message, = hello_world.send_message() print(my_message)

Run yarpserver from the command line. Now execute the Python script above (with Wrapyfi) twice, setting one instance to LISTEN = False and another to LISTEN = True. You can now type and return a message on the publisher's terminal and preview it within the listener's

• Request-Reply (REQ/REP): A requester sends a request to a responder, which responds to the request in a synchronous manner. e.g., with ROS

Without Wrapyfi

With Wrapyfi

# Just your usual Python class class HelloWorld(object): def send_message(self, a, b): msg = input("Type your message: ") obj = {"message": msg, "a": a, "b": b, "sum": a + b} return obj, hello_world = HelloWorld() while True: my_message, = hello_world.send_message(a=1, b=2) print(my_message)

from wrapyfi.connect.wrapper import MiddlewareCommunicator class HelloWorld(MiddlewareCommunicator): @MiddlewareCommunicator.register("NativeObject", "ros", "HelloWorld", "/hello/my_message", carrier="", should_wait=True) def send_message(self, a, b): msg = input("Type your message: ") obj = {"message": msg, "a": a, "b": b, "sum": a + b} return obj, hello_world = HelloWorld() LISTEN = True mode = "request" if LISTEN else "reply" hello_world.activate_communication(hello_world.send_message, mode=mode) while True: my_message, = hello_world.send_message(a=1 if LISTEN else None, b=2 if LISTEN else None) print(my_message)

Run roscore from the command line. Now execute the Python script above (with Wrapyfi) twice, setting one instance to LISTEN = False and another to LISTEN = True. You can now type and return a message on server's terminal and preview it within the client's. Note that the server's command line will not show the message until the client's command line has been used to send a request. The arguments are passed from the client to the server and the server's response is passed back to the client.

For more examples of usage, refer to the user guide. Run scripts in the examples directory for trying out Wrapyfi.

Supported Formats

Serializers

• JSON
• msgpack
• protobuf

Data Structures

Supported Objects by the NativeObject type include:

• NumPy Array | Generic
• PyTorch Tensor
• TensorFlow 2 Tensor
• JAX Tensor
• Trax Array
• MXNet Tensor
• PaddlePaddle Tensor
• pandas DataFrame | Series
• Pillow Image
• PyArrow Array
• CuPy Array
• Xarray DataArray | Dataset
• Dask Array | DataFrame
• Zarr Array | Group
• Pint Quantity
• Gmpy 2 MPZ
• MLX Tensor

Image

Supported Objects by the Image type include:

• NumPy Array [supports many libraries including scikit-image, imageio, Open CV, imutils, matplotlib.image, and Mahotas]

Sound

Supported Objects by the AudioChunk type include:

• Tuple(NumPy Array, int) [supports the sounddevice format]