cltl.backend-naoqi 0.0.dev12

Backend for Pepper robot

cltl-backend-naoqi

NaoQI Backend for the Pepper Robot.

Backend server

This repository provides a web server that supports connections of the client sources provided in the cltl.backend.source.client_source module of the Leolani Backend to the Pepper robot. The web server provides endpoints for Audio, Images as well as Text to Speech.

The web server can be run on the Pepper robot directly, or from a docker image that can be built from this repository. Mind that the Docker image will only run on Linux due to networking issues (see below).

Audio

The container exposes an endpoint at /audio (GET) that provides an audio stream. The format of the stream is indicated by the mine type indicated in the response. Currently, this will be chunked audio data with mime type audio/L16, i.e. raw 16bit audio. In the mime type header, additional parameters for rate, channels and frame_size are provided.

Images

The container exposes an endpoint at /video (GET) that provides an image captured at invocation time. The endpoint returns a JSON object corresponding to cltl.backend.api.camera.Image that contains the raw image data, the image dimensions and depth information. Image dimensions are also inlcude in the mime type header of the response.

Run the backend on the robot

To run the backend directly on the robot, the cltl.backend-naoqi package with its dependencies needs to be installed and run on the robot.

Package installation

On pepper, a rather old Python version is installed, which causes issues with pip. Also, root access of the nao user is restricted. pip can still be upgraded for the nao user using (run in a separate directory)

wget https://files.pythonhosted.org/packages/ca/1e/d91d7aae44d00cd5001957a1473e4e4b7d1d0f072d1af7c34b5899c9ccdf/pip-20.3.3.tar.gz
pip install --user pip-20.3.3.tar.gz

Since we do not want to interfere with the Python environment on pepper, we install the package in a virutal environment. For the installation the scripts/download_libs.sh and scripts/install_libs.sh scripts are provided. The can be downloaded via

    wget --no-check-certificate https://raw.githubusercontent.com/leolani/cltl-backend-naoqi/main/scripts/backend_naoqi.sh
    wget --no-check-certificate https://raw.githubusercontent.com/leolani/cltl-backend-naoqi/main/scripts/download_libs.sh
    wget --no-check-certificate https://raw.githubusercontent.com/leolani/cltl-backend-naoqi/main/scripts/install_libs.sh

First execute the download script and then the install script.

If this fails on the robot, first execute the download script on your local machine (it fails on the robot due to ssl issues). This will downlaod all necessary packages to a folder called lib/. Then copy this folder to the robot using scp:

    scp -r lib nao@<robot-ip>:<cltl-backend-install-dir>
    scp -r scripts nao@<robot-ip>:<cltl-backend-install-dir>

Then log in to the robot, navigate into the directory where the lib/ folder was uploaded, and run the scripts/install_libs.sh script. This will create a virtual environment and install the packages in lib/.

Running the backend

To run the backend we need to find an unused port that can be used by the web server and make sure the firewall on Pepper allows access it . By default the server uses port 8000, which typically is available.

To run the server, execute the scripts/backend_naoqi.sh script with one of the options start, run, state, or stop. The start option can be used to run the server in the background using nohup, i.e. you can disconnect from the robot after starting the server. The run option will start the script directly and is equivalent to

source venv/bin/activate
python -m cltl.naoqi --naoqi-ip 127.0.0.1 --port 8000

Configuration

Additional configuration parameters can be added when starting the web server, for a comprehensive list run

python -m cltl.naoqi --help

This configuration options can be either specified as command line options or environment variables.

Text to speech

The robot has Text to Speech (TTS) as built-in feature. To use TTS on the robot, send a POST requests to the text/ (POST) endpoint with the text as payload and mime type text/plain. The text posted to the robot can be annotated with animation instructions.

Backend Docker container

At the current moment running the Docker container on works on LINUX, as the --network host option for Docker is only available on LINUX and some services registered with the qi framework are not visible to the ALAudioDevice Module when we run inside the Docker container with a bridge network.

Build the backend Docker container

To build the docker image run

make docker

This will create a docker image with tag cltl-backend-naoqi. To verify the build was successful run

make test

Run the backend Docker container

To run the docker image use:

docker run --rm -it -e CLTL_NAOQI_IP="192.0.0.1" --network host cltl/cltl-backend-naoqi

It is mandatory to provide the IP of the Pepper robot in the CLTL_NAOQI_IP environment variable. Further configurations can be set through environment variables, for a list run

docker run --rm -it cltl/cltl-backend-naoqi python -m cltl.naoqi --help

Test if the backend is working

To test if the backend is working we can curl the REST API of the backend. The following tests require numpy and cv2 to be installed for python and the jq and sox command line applications to be installed. Alternatively redirect the output of the curl command to a file and check the content manually.

When testing /video, hit Enter on the image to close it. When testing /audio, mind the echo..

curl <backend_ip:port>/image
curl -N <backend_ip:port>/audio |  play -t raw -e signed -b 16 -c 1 -r 16000 -
curl -H "Content-Type: text/plain" --data "Hallo Stranger!" <backend_ip:port>/text

Unit test

The unit tests in this repository require the NAOqi SDK and Python 2.7 to be installed. They are, however, included in the Docker image provided by this repository and can be run with:

make test

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the MIT License. See LICENSE for more information.

Authors

• Taewoon Kim
• Thomas Baier
• Selene Báez Santamaría
• Piek Vossen