tabular2mcap 0.1.8

Convert tabular data (CSV, Parquet, Excel, etc.) to MCAP format with support for ROS2 and JSON schemas

Tabular2MCAP

Convert tabular data (CSV, Parquet, Excel, and more) to MCAP format with support for ROS2 and JSON schemas, enabling seamless integration with robotics workflows, data visualization in Foxglove Studio, and playback of sensor and navigation data.

Please ⭐ if this helps you today!

Features

• Multi-Format Support: Convert to ROS2 or JSON messages with support for standard message types (e.g., sensor_msgs/msg/NavSatFix, geometry_msgs/msg/TransformStamped, foxglove.LocationFix, and more)
• Configuration-Driven: YAML-based mapping with Jinja2 templates for flexible data transformation
• Batch Processing: Process multiple files and directories with a single command

Quick Start

Installation

# Basic installation (CSV, TSV, JSON, Pickle)
pip install tabular2mcap

# With optional formats
pip install tabular2mcap[parquet]      # Parquet files
pip install tabular2mcap[feather]      # Feather files
pip install tabular2mcap[orc]          # ORC files
pip install tabular2mcap[excel]        # Excel files (.xlsx)
pip install tabular2mcap[excel-legacy] # Excel files (.xls)
pip install tabular2mcap[xml]          # XML files
pip install tabular2mcap[all-formats]  # All formats

Basic Usage

tabular2mcap \
  -i /path/to/data/directory \
  -o output.mcap \
  -c config.yaml \
  -f converter_functions.yaml

Configuration System

The tool uses YAML configuration files to define how CSV files should be processed. These are:

• config.yaml: Defines file patterns and converter function mappings
• converter_functions.yaml: Contains Jinja2 templates for data transformation

Example Configurations

JSON Format:

# config.yaml
writer_format: "json"
file_mappings:
  - file_pattern: 'data.csv'
    converter_functions:
      - function_name: "row_to_foxglove_location_fix"
        schema_name: "foxglove.LocationFix"
        topic_suffix: "LocationFix"

ROS2 Format:

# config.yaml
writer_format: "ros2"
tabular_mappings:
  - file_pattern: '**/data.csv'
    converter_functions:
      - function_name: "row_to_nav_sat_fix"
        schema_name: "sensor_msgs/msg/NavSatFix"
        topic_suffix: "NavSatFix"

Tutorial

For detailed instructions on adding support for new types of CSV data, see the Tutorial in the documentation.

Supported By

This project is supported by Alloy - Search all your robot data in plain english. Alloy provides a platform to unify image, time-series, and log search for robotics teams.

License

GNU General Public License v3.0 - see LICENSE for details.

Contributing

See CONTRIBUTING.md for development setup, testing, and contribution guidelines.