HEX FLOW NODE MUJOCO

      

---

📖 Overview

What is hex_flow_node_mujoco

hex_flow_node_mujoco provides hex-flow nodes for running MuJoCo-based robot simulations. It currently supports two robot models: the Archer Y6 (single-arm manipulator) and the E3 Desktop (dual-arm manipulator with head camera). Each node wraps the MuJoCo physics engine behind a hex_flow_core node, publishing robot state and camera images at the configured rate over Zenoh via FlatBuffer messages from hex_util_msg, while accepting control commands from the network.

What problem it solves

• Simulation-as-a-Service: Turns MuJoCo simulation into a network-accessible node that publishes state and accepts control commands over Zenoh, enabling seamless integration with the hex-flow ecosystem.
• Rate-controlled state publishing: Decouples the MuJoCo simulation step from publish output, ensuring consistent state updates at a configurable rate regardless of simulation complexity.
• Multi-camera support: Supports synchronous color and depth image streaming from configurable camera sources (USB-style rendering or disabled), ideal for vision-based control pipelines.
• Flexible control modes: Accepts arm control commands in multiple modes (MIT, compensation, position, pose) and gripper control modes (MIT, position), matching the real-robot control interfaces.
• Integrated testing: Ships with terminal-based test nodes that send predefined control commands and display the resulting state — no additional tools needed.

Target users

• Engineers developing and testing motion control algorithms for HEXFELLOW robot systems in simulation.
• Researchers running simulation-based experiments (reinforcement learning, trajectory optimization, perception) before deploying to real hardware.
• Developers building on the hex-flow framework who need a working MuJoCo simulation layer.

Nodes

Node	Description	Publishes	Subscribes
hex-flow-mujoco-archer-y6	Archer Y6 single-arm MuJoCo simulation	arm_state, grip_state, obj_pose, color, depth	arm_ctrl, grip_ctrl, reset
hex-flow-mujoco-archer-y6-test	Terminal viewer & control for Archer Y6	arm_ctrl, grip_ctrl	arm_state, grip_state, obj_pose, color, depth
hex-flow-mujoco-e3-desktop	E3 Desktop dual-arm MuJoCo simulation	left_arm_state, right_arm_state, left_grip_state, right_grip_state, obj_pose, head_color, head_depth, left_color, left_depth, right_color, right_depth	left_arm_ctrl, right_arm_ctrl, left_grip_ctrl, right_grip_ctrl, reset
hex-flow-mujoco-e3-desktop-test	Terminal viewer & control for E3 Desktop	left_arm_ctrl, right_arm_ctrl, left_grip_ctrl, right_grip_ctrl	left_arm_state, right_arm_state, left_grip_state, right_grip_state, obj_pose, head_color, head_depth, left_color, left_depth, right_color, right_depth

📦 Installation

Requirements

• Python >= 3.10
• OS: Ubuntu (or other Linux with GPU-accelerated GLFW)
• Core dependencies:
  • hex_flow_core >= 0.0.0, < 0.1.0
  • hex_driver_mujoco >= 0.1.0, < 0.2.0
  • hex_util_msg >= 0.0.0, < 0.1.0
  • hex_util_runtime >= 0.0.0, < 0.1.0

Install hex-flow-cli

For Ubuntu or any Debian-based system, install Zenoh and hex-flow CLI:

sudo apt update
sudo apt install -y curl gpg

curl -L https://download.eclipse.org/zenoh/debian-repo/zenoh-public-key | sudo gpg --dearmor --yes --output /etc/apt/keyrings/zenoh-public-key.gpg
echo "deb [signed-by=/etc/apt/keyrings/zenoh-public-key.gpg] https://download.eclipse.org/zenoh/debian-repo/ /" | sudo tee -a /etc/apt/sources.list > /dev/null
sudo apt update
sudo apt install zenoh curl

curl -fsSL https://raw.githubusercontent.com/hexfellow/hex-flow/main/install.sh | sh

For other systems, please install zenohd yourself, then run the install script.

Install hex-flow-node-mujoco from PyPI

pip install hex_flow_node_mujoco

Install hex-flow-node-mujoco from source

We provide a venv.sh script to create a virtual environment with all dependencies installed. However, you need to install uv first. For uv installation, please refer to uv official installation guide.

curl -LsSf https://astral.sh/uv/install.sh | sh

Then you can use our venv.sh to create a virtual environment with all dependencies installed:

git clone https://github.com/hexfellow/hex_flow_node_mujoco.git
cd hex_flow_node_mujoco
./venv.sh

📑 Python Config API

The package provides helper functions that return NodeConfig objects for easy integration into your LaunchConfig.

from hex_flow_core import LaunchConfig
from hex_flow_node_mujoco import (
    default_mujoco_archer_y6_node,
    default_mujoco_archer_y6_test_node,
)

config = LaunchConfig(
    local_only=True,
    enable_tui=True,
    log_to_file=True,
    save_path="/tmp/mujoco_archer_y6_test.yml",
)

nodes = {
    "mujoco_archer_y6":
    default_mujoco_archer_y6_node(
        name="mujoco_archer_y6",
        state_rate=500,
        cam_rate=30,
        headless=False,
        state_buffer_size=200,
        cam_buffer_size=8,
        sens_ts=False,
        camera_type="usb",
        rate_hz=500,
        required=True,
        hidden=True,
    ),
    "test_mujoco_archer_y6":
    default_mujoco_archer_y6_test_node(
        name="test_mujoco_archer_y6",
        rate_hz=10,
        arm_ctrl_mode="pos",
        grip_ctrl_mode="pos",
        required=False,
    ),
}

config.set_nodes(nodes)
print(config.export())

default_mujoco_archer_y6_node

Parameter	Type	Default	Description
name	str	"mujoco_archer_y6"	Node name and remap prefix
state_rate	float	1000.0	Internal MuJoCo state update rate (Hz)
cam_rate	float	30.0	Camera rendering rate (Hz)
headless	bool	False	Run without GUI window
state_buffer_size	int	200	State ring buffer size
cam_buffer_size	int	8	Camera frame ring buffer size
sens_ts	bool	False	Use sensor timestamp instead of system time
camera_type	str	"usb"	Camera rendering type ("usb" or "empty")
rate_hz	float	500.0	Control command polling rate (Hz)
required	bool	True	Required for launch
hidden	bool	False	Hidden from TUI display
remap_dict	dict	None	Custom remap; auto-generated if None

default_mujoco_archer_y6_test_node

Parameter	Type	Default	Description
name	str	"test_mujoco_archer_y6"	Node name
rate_hz	float	10.0	Control & display rate (Hz)
arm_ctrl_mode	str	"pos"	Arm control mode ("mit", "pos", "pose")
grip_ctrl_mode	str	"pos"	Gripper control mode ("mit", "pos")
required	bool	False	Required for launch
hidden	bool	False	Hidden from TUI display
remap_dict	dict	None	Custom remap; defaults to {mujoco_name}/-prefixed topics
mujoco_name	str	"mujoco_archer_y6"	MuJoCo simulation node to subscribe to

default_mujoco_e3_desktop_node

Parameter	Type	Default	Description
name	str	"mujoco_e3_desktop"	Node name and remap prefix
state_rate	float	1000.0	Internal MuJoCo state update rate (Hz)
cam_rate	float	30.0	Camera rendering rate (Hz)
headless	bool	False	Run without GUI window
state_buffer_size	int	200	State ring buffer size
cam_buffer_size	int	8	Camera frame ring buffer size
sens_ts	bool	False	Use sensor timestamp instead of system time
head_cam_type	str	"empty"	Head camera type ("usb" or "empty")
left_cam_type	str	"empty"	Left arm camera type ("usb" or "empty")
right_cam_type	str	"empty"	Right arm camera type ("usb" or "empty")
rate_hz	float	500.0	Control command polling rate (Hz)
required	bool	True	Required for launch
hidden	bool	False	Hidden from TUI display
remap_dict	dict	None	Custom remap; auto-generated if None

default_mujoco_e3_desktop_test_node

Parameter	Type	Default	Description
name	str	"test_mujoco_e3_desktop"	Node name
rate_hz	float	10.0	Control & display rate (Hz)
arm_ctrl_mode	str	"pos"	Arm control mode ("mit", "pos", "pose")
grip_ctrl_mode	str	"pos"	Gripper control mode ("mit", "pos")
required	bool	False	Required for launch
hidden	bool	False	Hidden from TUI display
remap_dict	dict	None	Custom remap; defaults to {mujoco_name}/-prefixed topics
mujoco_name	str	"mujoco_e3_desktop"	MuJoCo simulation node to subscribe to

💡 Examples

Ready-to-run config scripts are provided in the example/ directory. Each script prints a launch YAML to stdout, intended for use with hexflow run:

Archer Y6

# 500 Hz control loop, 30 Hz camera, terminal state display at 10 Hz
hexflow run example/archer_y6_test.launch.py

E3 Desktop

# 500 Hz control loop, 30 Hz camera, dual-arm state display at 10 Hz
hexflow run example/e3_desktop_test.launch.py

YAML Examples

Archer Y6 (500 Hz control, 30 Hz camera)

nodes:
  - name: mujoco_archer_y6
    build: pip install hex_flow_node_mujoco
    run: hex-flow-mujoco-archer-y6
    required: true
    hidden: true
    remap:
      arm_state: mujoco_archer_y6/arm_state
      grip_state: mujoco_archer_y6/grip_state
      obj_pose: mujoco_archer_y6/obj_pose
      color: mujoco_archer_y6/color
      depth: mujoco_archer_y6/depth
      arm_ctrl: mujoco_archer_y6/arm_ctrl
      grip_ctrl: mujoco_archer_y6/grip_ctrl
      reset: mujoco_archer_y6/reset
    env:
      STATE_RATE: "500"
      CAM_RATE: "30"
      HEADLESS: "false"
      STATE_BUFFER_SIZE: "200"
      CAM_BUFFER_SIZE: "8"
      SEN_TS: "false"
      CAMERA_TYPE: "usb"
      RATE_HZ: "500"

  - name: test_mujoco_archer_y6
    build: pip install hex_flow_node_mujoco
    run: hex-flow-mujoco-archer-y6-test
    required: false
    hidden: true
    remap:
      arm_state: mujoco_archer_y6/arm_state
      grip_state: mujoco_archer_y6/grip_state
      obj_pose: mujoco_archer_y6/obj_pose
      color: mujoco_archer_y6/color
      depth: mujoco_archer_y6/depth
      arm_ctrl: mujoco_archer_y6/arm_ctrl
      grip_ctrl: mujoco_archer_y6/grip_ctrl
    env:
      RATE_HZ: "10"
      ARM_CTRL_MODE: "pos"
      GRIP_CTRL_MODE: "pos"

E3 Desktop (500 Hz control, 30 Hz camera, head camera enabled)

nodes:
  - name: mujoco_e3_desktop
    build: pip install hex_flow_node_mujoco
    run: hex-flow-mujoco-e3-desktop
    required: true
    hidden: true
    remap:
      left_arm_state: mujoco_e3_desktop/left_arm_state
      right_arm_state: mujoco_e3_desktop/right_arm_state
      left_grip_state: mujoco_e3_desktop/left_grip_state
      right_grip_state: mujoco_e3_desktop/right_grip_state
      obj_pose: mujoco_e3_desktop/obj_pose
      head_color: mujoco_e3_desktop/head_color
      head_depth: mujoco_e3_desktop/head_depth
      left_color: mujoco_e3_desktop/left_color
      left_depth: mujoco_e3_desktop/left_depth
      right_color: mujoco_e3_desktop/right_color
      right_depth: mujoco_e3_desktop/right_depth
      left_arm_ctrl: mujoco_e3_desktop/left_arm_ctrl
      right_arm_ctrl: mujoco_e3_desktop/right_arm_ctrl
      left_grip_ctrl: mujoco_e3_desktop/left_grip_ctrl
      right_grip_ctrl: mujoco_e3_desktop/right_grip_ctrl
      reset: mujoco_e3_desktop/reset
    env:
      STATE_RATE: "1000"
      CAM_RATE: "30"
      HEADLESS: "false"
      STATE_BUFFER_SIZE: "200"
      CAM_BUFFER_SIZE: "8"
      SEN_TS: "false"
      HEAD_CAM_TYPE: "usb"
      LEFT_CAM_TYPE: "empty"
      RIGHT_CAM_TYPE: "empty"
      RATE_HZ: "500"

  - name: test_mujoco_e3_desktop
    build: pip install hex_flow_node_mujoco
    run: hex-flow-mujoco-e3-desktop-test
    required: false
    hidden: true
    remap:
      left_arm_state: mujoco_e3_desktop/left_arm_state
      right_arm_state: mujoco_e3_desktop/right_arm_state
      left_grip_state: mujoco_e3_desktop/left_grip_state
      right_grip_state: mujoco_e3_desktop/right_grip_state
      obj_pose: mujoco_e3_desktop/obj_pose
      head_color: mujoco_e3_desktop/head_color
      head_depth: mujoco_e3_desktop/head_depth
      left_color: mujoco_e3_desktop/left_color
      left_depth: mujoco_e3_desktop/left_depth
      right_color: mujoco_e3_desktop/right_color
      right_depth: mujoco_e3_desktop/right_depth
      left_arm_ctrl: mujoco_e3_desktop/left_arm_ctrl
      right_arm_ctrl: mujoco_e3_desktop/right_arm_ctrl
      left_grip_ctrl: mujoco_e3_desktop/left_grip_ctrl
      right_grip_ctrl: mujoco_e3_desktop/right_grip_ctrl
    env:
      RATE_HZ: "10"
      ARM_CTRL_MODE: "pos"
      GRIP_CTRL_MODE: "pos"

Message Types (FlatBuffer)

All topics use FlatBuffer messages from hex_util_msg.

State Topics — Published by Simulation Nodes

{side}_arm_state — HexArmState

Field	Type	Description
ts_ns	int64	Timestamp in nanoseconds
jnt_pos	[float64]	Joint position array
jnt_vel	[float64]	Joint velocity array
jnt_eff	[float64]	Joint effort/torque array
pose_pos	[float64]	End-effector position (x, y, z)
pose_quat	[float64]	End-effector orientation (w, x, y, z)

Schema: msgs/msg_robot/arm_state.fbs

{side}_grip_state — HexGripState

Field	Type	Description
ts_ns	int64	Timestamp in nanoseconds
jnt_pos	[float64]	Gripper joint position
jnt_vel	[float64]	Gripper joint velocity
jnt_eff	[float64]	Gripper joint effort

Schema: msgs/msg_robot/grip_state.fbs

obj_pose — HexPoseState

Field	Type	Description
ts_ns	int64	Timestamp in nanoseconds
pose_pos	[float64]	Object position (x, y, z)
pose_quat	[float64]	Object orientation (w, x, y, z)

Schema: msgs/msg_robot/pose_state.fbs

{cam}_color / {cam}_depth — HexImgSimple

Field	Type	Description
ts_ns	int64	Timestamp in nanoseconds
encoding	HexImgEncoding	Image encoding (bgr8 or mono16)
width	uint16	Image width
height	uint16	Image height
data	[uint8]	Raw image pixel data (flattened)

For Archer Y6: topics color, depth. For E3 Desktop: topics head_color, head_depth, left_color, left_depth, right_color, right_depth.

Schema: msgs/msg_image/img_simple.fbs

Control Topics — Subscribed by Simulation Nodes

{side}_arm_ctrl — HexArmCtrl

Field	Type	Description
ts_ns	int64	Timestamp in nanoseconds
ctrl_mode	HexArmCtrlMode	Control mode enum (mit=0, comp=1, pos=2, pose=3, pos_plan=4, pose_plan=5)
jnt_pos	[float64]	Joint position target
jnt_vel	[float64]	Joint velocity target
jnt_eff	[float64]	Joint effort target
pose_pos	[float64]	End-effector position target (= pose mode)
pose_quat	[float64]	End-effector orientation target (= pose mode)
mit_tau	[float64]	MIT mode torque feedforward
mit_kp	[float64]	MIT mode stiffness gains
mit_kd	[float64]	MIT mode damping gains
lim_err	[float64]	Position error limit for safety
lim_vel	[float64]	Velocity limit
lim_acc	[float64]	Acceleration limit

Schema: msgs/msg_robot/arm_ctrl.fbs

{side}_grip_ctrl — HexGripCtrl

Field	Type	Description
ts_ns	int64	Timestamp in nanoseconds
ctrl_mode	HexGripCtrlMode	Control mode enum (mit=0, comp=1, pos=2, force=3)
jnt_pos	[float64]	Gripper joint position target
jnt_vel	[float64]	Gripper joint velocity target
jnt_eff	[float64]	Gripper joint effort target
grip_force	[float64]	Desired grip force (= force mode)
mit_tau	[float64]	MIT mode torque feedforward
mit_kp	[float64]	MIT mode stiffness gains
mit_kd	[float64]	MIT mode damping gains
lim_err	[float64]	Position error limit for safety

Schema: msgs/msg_robot/grip_ctrl.fbs

reset — Raw Bytes

Trigger for simulation reset. The simulation node resets the MuJoCo model to its initial state upon receiving any data on this topic.

Environment Variables

All Nodes

Variable	Type	Default	Description
HEX_FLOW_NODE_NAME	str	constructor arg	Overrides node name (handled by hex_flow_core)
HEX_FLOW_REMAP	str	{}	JSON dict for topic remapping (handled by hex_flow_core)
RUST_LOG	str	info	Log level for envlog

Simulation Nodes (archer-y6 / e3-desktop)

Variable	Type	Default	Description
STATE_RATE	float	1000.0	Internal state update rate (Hz)
CAM_RATE	float	30.0	Camera rendering rate (Hz)
HEADLESS	bool	false	Run without GUI window
STATE_BUFFER_SIZE	int	200	State ring buffer size
CAM_BUFFER_SIZE	int	8	Camera frame ring buffer size
SEN_TS	bool	false	Use sensor timestamp instead of system time
RATE_HZ	float	500.0	Control command polling rate (Hz)

Archer Y6 only

Variable	Type	Default	Description
CAMERA_TYPE	str	"usb"	Camera type ("usb" or "empty")

E3 Desktop only

Variable	Type	Default	Description
HEAD_CAM_TYPE	str	"empty"	Head camera type ("usb" or "empty")
LEFT_CAM_TYPE	str	"empty"	Left arm camera type ("usb" or "empty")
RIGHT_CAM_TYPE	str	"empty"	Right arm camera type ("usb" or "empty")

Test Nodes (archer-y6-test / e3-desktop-test)

Variable	Type	Default	Description
RATE_HZ	float	10.0	Control & display rate (Hz)
ARM_CTRL_MODE	str	"pos"	Arm control mode ("mit", "pos", "pose")
GRIP_CTRL_MODE	str	"pos"	Gripper control mode ("mit", "pos")

Architecture

Both Archer Y6 and E3 Desktop simulation nodes follow a similar pattern:

1. MuJoCo simulation callback — wraps hex_driver_mujoco driver (HexMujocoArcherY6Callback / HexMujocoE3DesktopCallback) which runs the physics engine at a configurable internal rate (STATE_RATE). The driver provides callbacks that deliver the latest simulation state and camera frames.

2. FlatBuffer serialization — on each state or camera callback, the node snapshots the data, builds a FlatBuffer message (HexArmState, HexGripState, HexPoseState, or HexImgSimple) with a nanosecond timestamp, and publishes the binary payload via hex_flow_core.Node.pub().

3. Control command dispatch — the node polls subscribed control topics (arm_ctrl, grip_ctrl) at the configured RATE_HZ, parses the FlatBuffer message, and dispatches the command to the appropriate driver method based on control mode (MIT, compensation, position, or pose).

4. Simulation reset — upon receiving data on the reset topic, the simulation is reset to its initial state.

5. Camera rendering — camera images are rendered from the MuJoCo scene at CAM_RATE. Color images use bgr8 encoding and depth images use mono16. The Archer Y6 has a single camera (color, depth); the E3 Desktop has three configurable cameras (head, left, right).

Test nodes subscribe to the simulation node's topics via hex_flow_core.Node.create_sub() and poll with Node.get(topic, latest=True) at a lower rate (default 10 Hz). They publish predefined control commands and display the received state on the terminal using ANSI escape sequences, providing a quick way to verify the simulation pipeline.

This decouples the high-frequency MuJoCo physics stepping from the network-facing publish and subscribe rates, allowing simulation to run at its natural frequency while providing consistent state updates and control handling at user-configured rates.

📄 License

Apache License 2.0. See LICENSE.

🌟 Star History

👥 Contributors