Omega-Prime: Data Model, Data Format and Python Library for Handling Ground Truth Road Traffic Data

Data Model, Format and Python Library for ground truth road traffic data containing information on dynamic objects, map and environmental factors optimized for representing urban traffic. The repository contains:

Data Model and Specification

see Data Model & Specification

• 🌍 Data Model: What signals exist and how these are defined.
• 🧾 Data Format Specification: How to exchange and store those signals.

Python Library

• 🔨 Create omega-prime files from many sources (see ./docs/notebooks/tutorial.ipynb):
  • ASAM OSI GroundTruth trace (e.g., output of esmini)
  • Table of moving object data (e.g., csv data)
  • ASAM OpenDRIVE map
  • LevelXData datasets through lxd-io
  • Extend yourself by subclassing DatasetConverter
  • Use omega-prime-ros to convert ROS 2 bag files into omega-prime
  • Use omega-prime-trajdata to convert motion prediction datasets into omega-prime
• 🗺️ Map Association: Associate Object Location with Lanes from OpenDRIVE or OSI Maps (see ./docs/notebooks/tutorial_locator.ipynb)
• 📺 Plotting of data: interactive top view plots using altair
• ✅ Validation of data: check if your data conforms to the omega-prime specification (e.g., correct yaw) using pandera
• 📐 Interpolation of data: bring your data into a fixed frequency
• 📈 Metrics: compute interaction metrics like PET, TTC, THW (see ./docs/notebooks/tutorial_metrics.ipynb) or apply data qualification metrics
  • Predicted and observed timegaps based on driving tubes (see ./omega_prime/metrics/analysis)
  • 2D-birds-eye-view visibility with omega-prime-visibility
  • Data qualification metrics assessing coverage, completeness and more. Use the CLI omega-prime qualify --help for further information.
• 🚀 Fast Processing directly on DataFrames using polars, polars-st
• ⌨️ CLI to convert, validate and visualize omega-prime files

The data model and format utilize ASAM OpenDRIVE and ASAM Open-Simulation-Interface GroundTruth messages. omega-prime sets requirements on presence and quality of ASAM OSI GroundTruth messages and ASAM OpenDRIVE files and defines a file format for the exchange and storage of these.

Omega-Prime is the successor of the OMEGAFormat. It has the benefit that its definition is directly based on the established standards ASAM OSI and ASAM OpenDRIVE and carries over the data quality requirements and the data tooling from OMEGAFormat. Therefore, it should be easier to incorporate omega-prime into existing workflows and tooling.

To learn more about the example data read example_files/README.md. Example data was taken and created from esmini.

Installation

pip install omega-prime

Usage

A detailed introduction to the features and usage can be found in ./docs/notebooks/tutorial.ipynb

Create an omega-prime file from an OSI GroundTruth message trace and an OpenDRIVE map:

import omega_prime

r = omega_prime.Recording.from_file("example_files/pedestrian.osi", map_path="example_files/fabriksgatan.xodr")
r.to_mcap("example.mcap")

If you want to create an OSI trace on your own in python, check out the python library betterosi.

Read and plot an omega-prime file:

r = omega_prime.Recording.from_file("example.mcap")
ax = r.plot()

Convert Existing Datasets to omega-prime

LevelXData

You can convert data from LevelXData to omega-prime. Under the hood lxd-io is used to perform the conversion.

from omega_prime.converters import LxdConverter

converter = LxdConverter("./exiD-dataset-v2.0", "./exiD-as-omega-prime", n_workers=4)
# convert the dataset and store the omega-prime files in the new directory
converter.convert()
# access Recordings directly without storing them
iterator_of_recordings = converter.yield_recordings()

or with omega-prime from-lxd ./exiD-dataset-v2.0 ./exiD-as-omega-prime --n-workers=4.

Tested with exiD-v2.0, inD-v1.1, highD-v1.0 (highD does not provide an ASAM OpenDRIVE map).

File Format

Based on MCAP, ASAM OSI and ASAM OpenDRIVE the ASAM OSI GroundTruth messages and ASAM OpenDRIVE map are packaged as shown in the following figure.

Visualization in Lichtblick

You can view omega-prime files in Lichtblick (fork of foxglove) using the plugins asam-opendrive-converter and asam-osi-converter

Lichtblick does not apply the projection metadata embedded in OSI GroundTruth. Create a separate file with the projected coordinates materialized before opening it:

omega-prime materialize-projection input.mcap output.mcap --flatten-map

--flatten-map additionally removes OpenDRIVE elevation and banking. Omit it when object Z coordinates and the map use the same height reference.

Acknowledgements

This package is developed as part of the SYNERGIES project.

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor the granting authority can be held responsible for them.

Notice