icoapi 0.1.2

A REST and WebSocket API for the ICOtronic system using the Python FastAPI library

ICOapi

A REST and WebSocket API using the Python FastAPI library. You can find the official documentation here.

We currently support all operating systems which can run Python 3.10. and use a CAN interface properly

When the API is running, it hosts an OpenAPI compliant documentation under /docs, e.g. under localhost:33215/docs.

Hardware

This API is designed to interact with the ICOtronic system and thus only reasonably works with this system connected.

To get a complete experience, even for development, you need:

• A CAN interface (usually either PCAN-USB or the RevPi CAN Module)
• The proper drivers

Linux

On Linux, the API (rather: the underlying CAN library) requires:

• The proper driver for your CAN device (PCAN-USB if used)
• The CAN port set up as described in this guide
  • Including the setup for systemd-networkd!

Installation for Development

This repository can be setup manually, installed as a system service on Linux-based systems or deployed using Docker on Linux-based systems.

If none of the versions for deploying properly (see chapter Run) work for you, you can always "deploy" by cloning this repository and running the Python script manually.

Prerequisites

• Python 3.10+, from the official Python Website
• Poetry
• Support for Python virtual environments (recommended)

Clone the repository and navigate into it to set up your virtual environment:

git clone ... && cd ...

python -m venv .venv

source ./.venv/bin/activate on Linux or .\.venv\Scripts\activate on Windows

Then run the following command to get up and running:

poetry lock && poetry install --all-extras

Once you have that run the API:

poetry run python3 icoapi/api.py

Run

Proper deployment (automatic restart, etc.) can be done using the system service installation or Docker.

System Service Installation (Linux)

For Linux, there is an installation script which sets the directory for the actual installation, the directory for the systemd service and the used systemd service name. The (sensible) defaults are:

SERVICE_NAME="icoapi"
INSTALL_DIR="/etc/icoapi"
SERVICE_PATH="/etc/systemd/system"

Run the script to install normally:

./install.sh

Or, if you want to delete existing installations and do a clean reinstall, add the --force flag:

./install.sh --force

Docker (Linux)

You can use our Dockerfile to build a Docker image for the API:

docker build -t icoapi .

To run a container based on the image you can use the following command:

docker run --network=host icoapi

Note: The option --network=host is required to give the container access to the CAN adapter. As far as we know using the CAN adapter this way only works on a Linux host. For other more secure options to map the CAN adapter into the container, please take a look at:

• the documentation of the ICOtronic library, and
• the article “SocketCAN mit Docker unter Linux”.

Environment Variables

The application expects a .env file in one of three locations, each one being the fallback for the location before. The respective function is written as:

def load_env_file():
    # First try: local development
    env_loaded = load_dotenv(os.path.join(os.getcwd(), "config", ".env"), verbose=True)
    if not env_loaded:
        # Second try: configs directory
        logger.warning(f"Environment variables not found in local directory. Trying to load from app data: {get_config_dir()}")
        env_loaded = load_dotenv(os.path.join(get_config_dir(), ".env"), verbose=True)
    if not env_loaded and is_bundled():
        # Third try: we should be in the bundled state
        bundle_dir = sys._MEIPASS
        logger.warning(f"Environment variables not found in local directory. Trying to load from app data: {bundle_dir}")
        env_loaded = load_dotenv(os.path.join(bundle_dir, "config", ".env"), verbose=True)
    if not env_loaded:
        logger.critical(f"Environment variables not found")
        raise EnvironmentError(".env not found")

1. For local development: the .env file is under /config/.env
2. For normal usage, the file is in the user_data_dir
3. When no environment variable file was found, we check the bundle directory from the pyinstaller for the bundled file

This means that the .env file is bundled at compile-time and if the user has not ever run the software or deleted the user_data_dir we can take it as a fallback.

All variables prefixed with VITE_ indicate that there is a counterpart in the client side environment variables. This is to show that changes here most likely need to be propagated to the client (and electron wrapper, for that matter).

Client/API Connection Settings

These settings determine all forms of client/API communication details.

The main REST API is versioned, does NOT use SSL at the moment and has certain origins set as secure for CORS.

VITE_API_PROTOCOL=http
VITE_API_HOSTNAME="0.0.0.0"
VITE_API_PORT=33215
VITE_API_VERSION=v1
VITE_API_ORIGINS="http://localhost,http://localhost:5173,http://localhost:33215,http://127.0.0.1:5173"

The WebSocket is for streaming data. It only requires a VITE_API_WS_PROTOCOL variable akin to VITE_API_PROTOCOL which decided between SSL or not, and how many times per second the WebSocket should send data.

VITE_API_WS_PROTOCOL=ws
WEBSOCKET_UPDATE_RATE=60

File Storage Settings

These settings determine where the measurement and configuration files are stored locally.

VITE_APPLICATION_FOLDER=ICOdaq

VITE_APPLICATION_FOLDER expects a single folder name and locates that folder under a certain path. We use the user_data_dir() from the package platformdirs to simplify this. The system always logs which folder is used for storage.

Logging Settings

LOG_LEVEL=DEBUG
LOG_USE_JSON=0
LOG_USE_COLOR=1
LOG_PATH="C:\Users\breurather\AppData\Local\icodaq\logs"
LOG_MAX_BYTES=5242880
LOG_BACKUP_COUNT=5
LOG_NAME_WITHOUT_EXTENSION=icodaq
LOG_LEVEL_UVICORN=INFO

LOG_LEVEL is one of DEBUG, INFO, WARNING, ERROR, CRITICAL

LOG_USE_JSON formats the logs in plain JSON if set to 1

• useful for production logs

LOG_USE_COLOR formats the logs in color if set to 1

• useful for local development in a terminal

LOG_PATH overrides the default log location as an absolute path to a directory

• You need to have permissions
• The defaults are:
  • Windows: AppData/Local/icodaq/logs
  • Linux/macOS: ~/.local/share/icodaq/logs

LOG_NAME_WITHOUT_EXTENSION sets the name of the logfile. Without any file extension.

LOG_MAX_BYETS and LOG_BACKUP_COUNT determine the maximum size and backup number of the logs.

LOG_LEVEL_UVICORN controls the log level for the uvicorn webserver logging.

Configuration Files

The API currently works with 3 configuration files in the .yaml format.

When the API is run, it checks for the availability of these files in the <user_data_dir> / config. If the files are not there, the defaults from the compile time are used.

You can find the default files for all three types under /config.

Configuration File Header

In each configuration file there must be a header containing information on the file and schema.

info:
  schema_name: sensors_schema
  schema_version: 0.0.1
  config_name: General Purpose Sensor File
  config_date: '2025-10-07T13:52:40+0200'
  config_version: 0.0.1

The above section is exemplary for a sensor configuration file.

Configuration File 1: Sensors

The internal library starts the measurement based on selected channels. It is up to the user to know which channels are connected to which sensors currently.

To help this selection and make using the system easier, a layer of abstraction is present in this API and thus in the client and ICOdaq software packages.

Data Structure

Withing the sensors.yaml file, two separate areas exist. One contains the sensor information and one the configurations which reference the sensors. Additionally, a field for the default configuration exists. The file then looks like this:

info:
  ...
sensors:
- ...
- ...

sensor_configurations:
- ...
- ...

default_configuration_id: 

Sensor Data Structure

The sensors (which are written to the *.hdf5 file when used) are defined as:

- name: Acceleration 100g
  offset: -125.0
  phys_max: 100.0
  phys_min: -100.0
  scaling_factor: 75.75757575757575
  sensor_id: acc100g_01
  sensor_type: ADXL1001
  unit: g
  dimension: Acceleration
  volt_max: 2.97
  volt_min: 0.33

This example defines the mainly used +-100g acceleration sensor in the X axis.

Note that the field sensor_id is what the API uses to identify the sensor for usage.

Sensor Configuration Data Structure

This is what actually affects the client. Configurations are what the user can choose from and what determines which sensors and channels a user can select for measurement.

The data is structured as follows:

- configuration_id: singleboard_GYRO
  configuration_name: GYRO
  channels:
    1:  { sensor_id: acc100g_01 }
    6:  { sensor_id: photo_01 }
    8:  { sensor_id: gyro_01 }
    10: { sensor_id: vbat_01 }

The configuration_id is what the client-side .env file can set to load as a default for tools.

The configuration_name is displayed as the client.

The mapping of sensors follows the schema of <channel>: { sensor_id: <sensor_id> }.

The default_configuration_id has one of the configuration_id set.

Config File 2: Metadata

To support the usage of arbitrary metadata when creating measurements, a configuration system has been set up. This system starts as an Excel file in which all metadata fields are defined. This file is then parsed into a YAML file, from which it can be used further.

The complete metadata logic can be found in the ICOweb repository.

The metadata is split into two parts:

• the metadata entered before a measurement starts (pre_meta)
• the metadata entered after the measurement has been ended (post_meta)

This ensures that common metadata like machine tool, process or cutting parameters are set beforehand while keeping the option to require data after the fact, such as pictures or tool breakage reports.

The pre-meta is sent with the measurement instructions while the post-meta is communicated via the open measurement WebSocket.

Configuration File 3: Dataspace

This file sets the dataspace connection settings if required. It simply holds all the relevant information as:

connection:
  enabled: False
  username: myUser
  password: strongPw123!
  bucket: common
  bucket_folder: default
  protocol: https
  domain: trident.example.com
  base_path: api/v1

All relevant fields are strings without any / before or after the value. This means that for the given example a complete endpoint would be:

https://trident.example.com/api/v1/<endpoint>

And the relevant storage would be in the folder default of the bucket common.

Measurement Value Conversion / Storage

The used ICOc library streams the data as unsigned 16-bit integer values. To get the actual measured physical values, we go through two conversion steps:

Step 1: 16-bit ADC value to Voltage

The streamed uint16 is a direct linear map from 0 - 2^16 to 0 - V_ref of the used ADC. This means we can reverse the conversion by inverting the linear map.

We will define the coefficients k1 and d1 as the factor and offset of going from bit-value to voltage respectively.

As the linear map is direct and without an offset, we can set:

d1 = 0
k1 = (V_ref)/(2^16) in Volt

The first conversion only depends on the used reference voltage.

Step 2: Voltage to Physical Value

Each used sensor has a datasheet and associated linear coefficients to get from voltage output to the measured physical values.

We will define k2 and d2 as the linear coefficients of going from voltage to physical measurement.

The API now accepts a sensor_id which can be used to choose a unique sensor for the conversion and has the current IFT channel-sensor-layout as defaults.

Test

Note: Running the tests (successfully) requires that

• you connected a STU to your test system and
• at least one sensor device (e.g. STH) is available.

poetry run pytest

Development Guidelines

These guidelines are a work-in-progress and aim to explain development decisions and support consistency.

Logging

The application is set up to log everything. This is how the logging is set up.

Guidelines

• Log only after success
• Don't log intent, like "Creating user..." or "Initializing widget..." unless it's for debugging.
• Do log outcomes, like "User created successfully." — but only after the operation completes without error.
• Avoid logging in constructors unless they cannot fail
  • Prefer logging in methods that complete the actual operation,
  • or use a factory method to wrap creation and success logging.

Levels

Action	Log Level	Description (taken from Python docs)
Starting a process / intention	DEBUG	Detailed information for diagnosing problems. Mostly useful for developers.
Successfully completed action	INFO	For confirming that things are working as expected.
Recoverable error / edge case	WARNING	Indicates something unexpected happened or could cause problems later.
Expected failure / validation	ERROR	Used for serious problems that caused a function to fail.
Critical Failure / unrecoverable	CRITICAL	For very serious errors. Indicates a critical condition — program may abort.
Unexpected exception (with trace)	logger.exception()	Serious errors, but the exception was caught.

Release

Note: In the text below we assume that you want to release version <VERSION> of the package. Please just replace this version number with the version that you want to release (e.g. 0.2.0).

1. Make sure that all the checks and tests work correctly locally

   make
   

2. Make sure all workflows of the CI system work correctly

3. Release a new version on PyPI:

   1. Increase version number
   2. Add git tag containing version number
   3. Push changes

   poetry version <VERSION>
   export icoapi_version="$(poetry version -s)"
   git commit -a -m "Release: Release version $icoapi_version"
   git tag "$icoapi_version"
   git push && git push --tags
   

4. Open the release notes for the latest version and create a new release

   1. Paste them into the main text of the release web page
   2. Insert the version number into the tag field
   3. For the release title use “Version ”, where <VERSION> specifies the version number (e.g. “Version 0.2”)
   4. Click on “Publish Release”

   Note: Alternatively you can also use the gh command:

   gh release create
   

   to create the release notes.

Example Requests

Note: The sample requests below use the command line version of httpie

Get list of available sensor devices:

http 'http://localhost:33215/api/v1/sth'

Example output:

[
    {
        "device_number": 0,
        "mac_address": "08-6B-D7-01-DE-81",
        "name": "Test-STH",
        "rssi": -44
    }
]

Connect to available sensor device:

http PUT 'http://localhost:33215/api/v1/sth/connect' mac_address='08-6B-D7-01-DE-81'

Check if the STU is connected to the sensor device:

http POST 'http://localhost:33215/api/v1/stu/connected' name='STU 1'

Disconnect from sensor device:

http PUT http://localhost:33215/api/v1/sth/disconnect