mujoco-usd-converter

Overview

A MuJoCo to OpenUSD Data Converter

Important: This is currently an Alpha product. See the CHANGELOG for features and known limitations.

Key Features:

• Converts an input MJCF file into an OpenUSD Layer
• Supports data conversion of visual geometry & materials, as well as the bodies, collision geometry, sites, joints, and actuators necessary for kinematic simulation.
• Available as a python module or command line interface (CLI).
• Creates a standalone, self-contained artifact with no connection to the source MJCF, OBJ, or STL data.
  • Structured as an Atomic Component
  • Suitable for visualization & rendering in any OpenUSD Ecosystem application.
  • Suitable for import & simulation directly in MuJoCo Simulate.

This project is part of Newton, a Linux Foundation project which is community-built and maintained.

Menagerie Benchmarks

We run regular benchmarks on the MuJoCo Menagerie. See the latest results here.

Implementation Details & Dependencies

Specific implementation details are based on the "MJC to USD Conceptual Data Mapping" document, which is a collaboration between Google DeepMind and NVIDIA. This document will be made public once the project moves out of the alpha phase.

One important detail is that this document recommends nested rigid bodies within articulations, as it more faithfully matches the kinematic tree in MuJoCo and meets the needs of reduced coordinate solvers. Support for nested bodies in UsdPhysics is fairly new (as of USD 25.11), and some existing applications may not support this style of nesting.

The output asset structure is based on NVIDIA's Principles of Scalable Asset Structure in OpenUSD.

The implementation also leverages the following dependencies:

• NVIDIA's OpenUSD Exchange SDK to author consistent & correct USD data.
• Pixar's OpenUSD python modules & native libraries (vendored via the usd-exchange wheel).
• Google DeepMind's mujoco python module for parsing MJCF into MjSpec
• A codeless version of the MjcPhysics USD schema from MuJoCo to author the MuJoCo specific Prims, Applied APIs, and Attributes.
• tinyobjloader and numpy-stl for parsing any mesh data referenced by the input MJCF datasets.

Get Started

To start using the converter, install the python wheel into a virtual environment using your favorite package manager:

python -m venv .venv
source .venv/bin/activate
pip install mujoco-usd-converter
mujoco_usd_converter /path/to/robot.xml /path/to/usd_robot

See mujoco_usd_converter --help for CLI arguments.

Alternatively, the same converter functionality can be accessed from the python module directly, which is useful when further transforming the USD data after conversion.

import mujoco_usd_converter
import usdex.core
from pxr import Sdf, Usd

converter = mujoco_usd_converter.Converter()
asset: Sdf.AssetPath = converter.convert("/path/to/robot.xml", "/path/to/usd_robot")
stage: Usd.Stage = Usd.Stage.Open(asset.path)
# modify further using Usd or usdex.core functionality
usdex.core.saveStage(stage, comment="modified after conversion")

Loading the USD Asset

Once your asset is saved to storage, it can be loaded into an OpenUSD Ecosystem application, including a custom build of MuJoCo itself.

We recommend starting with usdview, a simple graphics application to confirm the visual geometry & materials are working as expected. You can inspect any of the USD properties in this application, including the UsdPhysics and MjcPhysics properties.

Tip: OpenUSD Exchange Samples provides ./usdview.sh and .\usdview.bat commandline tools which bootstrap usdview with the necessary third party dependencies.

However, you cannot start simulating in usdview, as there is no native simulation engine in this application.

To simulate this asset in Newton, call newton.ModelBuilder.add_usd() to parse the asset and add it to your Newton model.

It is also possible to simulate this asset directly in MuJoCo itself.

Simulating in other UsdPhysics enabled products (e.g. NVIDIA Omniverse, Unreal Engine, etc) may provided mixed results. The MJC physics data is structured hierarchically, which maximal coordinate solvers often do not support. Similarly, many of the important simulation settings are authored via MjcPhysics schemas, which is a USD plugin developed by Google DeepMind, that needs to be deployed & supported for import by the target runtime. In order to see faithful simulation in these applications, the USD asset will need to be modified to suit the expectations of each target runtime.

Loading USD in MuJoCo Simulate

Loading any USD Layer into MuJoCo Simulate requires a USD enabled build of MuJoCo (i.e. built from source against your own OpenUSD distribution).

Important : USD support in MuJoCo is currently listed as experimental

To build MuJoCo with USD support, follow the usual CMake build instructions & provide the USD_DIR argument when configuring cmake. If you do not have a local USD distribution you will need to build or acquire one.

Tip: OpenUSD Exchange provides a commandline tool to acquire many precompiled distributions of OpenUSD across several platforms & python versions. See the install_usdex documentation for details.

Once MuJoCo is compiled, you can launch the ./bin/simulate app and drag & drop your USD asset into the viewport. The robot should load & simulate just as if you were using the original MJCF dataset.

Contribution Guidelines

Contributions from the community are welcome. See CONTRIBUTING.md to learn about contributing via GitHub issues, as well as building the project from source and our development workflow.

General contribution guidelines for Newton repositories are available here.

Community

For questions about this mujoco-usd-converter, feel free to join or start a GitHub Discussions.

For questions about OpenUSD Exchange SDK, use the USD Exchange GitHub Discussions.

For questions about MuJoCo or the MjcPhysics USD Schemas, use the MuJoCo Forums.

For general questions about OpenUSD itself, use the Alliance for OpenUSD Forum.

By participating in this community, you agree to abide by the Linux Foundation Code of Conduct.

References

• MuJoCo Docs
• NVIDIA OpenUSD Exchange SDK Docs
• OpenUSD API Docs
• OpenUSD User Docs
• NVIDIA OpenUSD Resources and Learning

License

The mujoco-usd-converter is provided under the Apache License, Version 2.0, as is the OpenUSD Exchange SDK and MuJoCo.