simcheck

Does your robot or object asset actually behave in simulation?

A URDF or MJCF file can load without a single warning and still be broken:

• The drawer can be jammed inside a convex hull.
• The mug can be a solid cylinder.
• A door can swing through its own cabinet.
• Joint limits can be silently ignored.
• The inertia tensor can be thrown away by the engine.

simcheck loads the asset in MuJoCo and PyBullet and runs it. It reports what each engine actually does with the file, and why.

$ simcheck tests/assets/cabinet_mesh.urdf
tests/assets/cabinet_mesh.urdf
  FAIL  score 94.6/100   1 error, 4 warnings, 2 notes   [mujoco, pybullet; 0.3 s]
  ERRO engine.rest_contacts [mujoco]: 'carcass' and 'drawer' overlap by 45.0 mm at rest in mujoco; the authored geometry doesn't overlap there: it is mujoco's convex hull of 'carcass', so split that mesh into convex parts (CoACD); they are parent and child, but MuJoCo still collides them because the parent is welded to the world, so joint 'drawer_slide' is braked by this contact
  WARN engine.collision_shape [mujoco]: mujoco treats 100% of the empty space inside 'carcass' as solid (1.93e+05 cm^3); objects can't go in. Split the mesh into convex parts (CoACD, V-HACD).
  ...
$ simcheck fix tests/assets/cabinet_mesh.urdf -o fixed/cabinet.urdf
  fixed: link 'carcass': concave collision mesh (hull 5.6x its volume) split into 58 convex parts
before: FAIL 94.6  (1 error, 4 warnings)
after:  PASS 100  (0 errors, 0 warnings)

Install

pip install "simcheck[all]"

Or from source:

git clone https://github.com/jashshah999/simcheck
pip install -e "simcheck[all]"

• Requirements: Python 3.10+. MuJoCo, trimesh and the other dependencies install automatically.
• Extras:
  • pybullet: the second engine.
  • fix: CoACD, needed by simcheck fix for convex decomposition.
  • all: both of the above.
• A plain install (pip install simcheck) checks in MuJoCo only.

Use

simcheck path/to/robot.urdf                 # one file
simcheck assets/ --html report.html         # a folder, with an HTML report
simcheck model.xml --engines mujoco         # MJCF, one engine
simcheck mug.urdf --free-base --category mug   # a loose object: also drop it on a floor
simcheck scan dataset/ -o results.jsonl     # thousands of files, parallel, resumable
simcheck fix cabinet.urdf                   # write cabinet.fixed.urdf and re-check it
simcheck list-checks -v                     # what every check does

Exit codes:

• check: 1 when any asset has an error. --fail-on warning makes warnings count too, and --fail-on never always exits 0.
• scan: exits 0 unless you pass --fail-on.
• fix: 1 if the repaired file still has an error.
• Every command exits 2 on a usage mistake, such as a missing file, an unknown engine or a misspelled option.

Output files: --json writes an object for one file and a list for several.

From Python:

import simcheck

report = simcheck.check("cabinet.urdf", engines=("mujoco", "pybullet"))
print(report.verdict, report.score)
for f in report.findings:
    print(f.severity, f.check, f.engine, f.message)
report.write_json("report.json")

In CI

- uses: jashshah999/simcheck@v0.1.0
  with:
    paths: assets/
    fail-on: error        # or warning
    args: --config simcheck.toml

The action does three things:

• Annotates the offending files in the pull request.
• Writes a summary table to the job page.
• Uploads nothing anywhere; reports stay in the workspace (simcheck-report.json, simcheck-report.html).

What it checks

35 checks in all; the full list with descriptions is in docs/checks.md, and docs/how-it-works.md explains the method.

Robots (files with actuators or transmissions, or robot-like link names) and objects are judged slightly differently: a hollow arm link or an arm that can fold into its own base is normal for a robot but a defect in a cabinet. Override with --robot / --object.

Score:

• The share of applicable checks passed; warnings count half.
• Any error makes the verdict FAIL.
• Any warning makes it WARN.

What engines do silently

The engine checks exist because each of these behaviours makes a valid-looking file behave differently than written. Each one was reproduced on MuJoCo 3.15 and PyBullet 3.2.7.

MuJoCo:

• Meshes collide as their convex hull, even on static bodies. Cabinets, drawers, mugs and bins become solid blocks.
• Parent-child contacts are skipped, except when the parent is welded to the world. A drawer inside a convexified fixed cabinet is therefore pushed out at the first step, while the same drawer in a free cabinet silently passes through it.
• A URDF joint with lower > upper or lower == upper becomes unlimited. PyBullet also leaves lower > upper unlimited but locks lower == upper.
• Two mesh files with the same file name in different folders become one mesh.
• A URDF root link without a joint is fused into the world, so a loaded object cannot move.
• It resets the simulation when it diverges. simcheck disables that, so blow-ups are visible.

PyBullet:

• The URDF inertia tensor is ignored unless URDF_USE_INERTIA_FROM_FILE is set; it is recomputed from the bounding box.
• A link without <inertial> gets 1 kg.
• Every joint starts with a velocity motor that holds it in place.
• Self-collision is off by default.

simcheck runs each engine in an "honest" configuration that turns these conveniences off, then reports where defaults would change the result.

Fixing

simcheck fix asset.urdf writes a repaired copy (asset.fixed.urdf, with new meshes in asset.fixed_meshes/) and re-checks it. It works on URDF files. It applies only mechanical fixes:

• decompose: concave collision meshes are split into convex parts with CoACD. It refines automatically until no other part of the asset sits inside the new shapes.
• collision: collision geometry is added from the visual for parts that have none.
• inertia: missing, zero or non-physical inertia is recomputed from geometry. It keeps the mass when it is valid, otherwise uses --density.
• limits: swapped limits are swapped back.
• axes: joint axes are normalized.

Problems that need a human decision are listed under "left", not guessed: a wrong axis direction, a hinge in the wrong place, an implausible mass, a zero effort limit. Convex decomposition of a large mesh can take a minute.

Leaderboard

leaderboard/ scores 1,868 assets from 15 public sources:

What it found:

• Generated assets rarely carry real physics. Of 1,000 Articraft-10K assets, 699 have no collision geometry. Of the other 301, 292 have moving parts with no <inertial>, so MuJoCo invents a mass at the density of water and PyBullet uses 1 kg and 1 kg m^2. In LAM it is 267 of 271, across all seven LLM backbones.
• Convex hulls create collisions nobody authored. 127 of 301 Articraft assets start with parts overlapping in MuJoCo. In 50 of them the authored meshes don't overlap; MuJoCo's convex hulls do. The same effect jams the drawers of 10 of the 11 URDFormer cabinets.
• Placeholder inertia is common. 124 of 130 EmbodiedGen objects and 60 of 78 YCB URDFs use the same inertia on every axis whatever their size.
• Visual meshes add mass. All 80 sampled Google Scanned Objects and all 49 RoboCasa fixtures get mass from visual-only geoms (one freezer drawer: 895 of its 956 kg).
• Curated robots have real issues too. Kinova Gen3's base and shoulder overlap by 12 mm, which MuJoCo turns into a brake on joint 1; Talos's default pose puts its grippers inside its thighs.

Every finding category was audited by opening the flagged files; leaderboard/README.md has the method and how to reproduce it, and leaderboard/stats.py recomputes every number above from the results.

Benchmark

simcheck bench assets/ plants 17 kinds of defect into every asset and reports which ones are caught. Examples:

• millimetre units
• reversed axis
• swapped limits
• broken inertia
• grams as kilograms
• missing collision
• offset collision
• centre of mass off the part
• broken mesh path
• joint inside its parent

On 52 real assets (YCB, EmbodiedGen, robot_descriptions and the test fixtures), it caught 631 of 655 planted defects (96%): all 536 on objects and 95 of 119 on robots. Most robot misses are deliberate: missing or offset collision geometry is a note on robots, and a reversed axis on a joint with symmetric limits is invisible geometrically. See leaderboard/BENCH.md.

Configuration

simcheck reads simcheck.toml, or a [tool.simcheck] table in pyproject.toml, from the current directory. You can also pass one with --config.

engines = ["mujoco", "pybullet"]
disable = ["geometry.visual_match"]   # check ids from `simcheck list-checks`
kind = "auto"                         # "robot", "object" or "auto"
fixed_base = true                     # omit to let simcheck decide
package_paths = { my_robot = "../my_robot_description" }   # where package://my_robot lives
seed = 0

[thresholds]
contact_error = 0.01      # m of penetration that counts as an error
convexity_warn = 1.5      # hull volume / mesh volume
density_warn_low = 20.0   # kg/m^3

Thresholds: every threshold is in simcheck/config.py. A misspelled key is reported with the closest valid name.

Size priors (category):

• Set a category per run with --category, or in the config file.
• Otherwise simcheck guesses one from the file and folder names: Microwave081/model.xml is a microwave.
• A guessed category only adds notes unless the size is off by 10x or more.
• Known categories: airship, blimp, bottle, box, bridge, bucket, cabinet, chair, clock, conveyor, crane, dishwasher, door, drawbridge, drawer, excavator, faucet, ferris, gate, helicopter, kettle, lamp, laptop, lighthouse, microwave, mug, oven, pen, pot, refrigerator, safe, scissors, table, toilet, tower, trashcan, washingmachine, window, windmill.

Limitations

• Formats:
  • URDF and MJCF only; USD is not read yet.
  • PyBullet reads URDF only, so MJCF files are checked in MuJoCo alone.
• Engines: Newton and Genesis are not backends yet.
• Platforms: tested on Linux with Python 3.10, 3.11 and 3.13. Engines run in forked child processes, so a native crash or hang costs one result. Windows has no fork, so there the engines run in-process and a native crash ends the run.
• Speed: most assets take 1-10 s. Large robots with many detailed meshes can take a few minutes; scan runs assets in parallel.
• What the checks test: the asset against itself and against physics, not against the real object. A cabinet that is internally consistent but twice the real size only shows up through a category size prior.
• Heuristics: some checks are heuristics with thresholds you can change:
  • the hollow-shell density rule
  • the pass-through share
  • the cavity-fill share
• Gaps: soft bodies, cloth, cables, fluids and tendons are not checked.

Related tools

• NVIDIA SimReady: a spec and benchmark for USD assets in Isaac and PhysX.
• simcompat: compares MuJoCo and PyBullet imports of URDF robots against Pinocchio, without contact.
• simready-audit: checks rigid USD/MJCF objects in MuJoCo.
• Static linters: check_urdf, urdf_parser_py, yourdfpy and others.

simcheck's focus:

• articulated objects and robots in URDF and MJCF;
• contact behaviour across engines;
• measuring the collision shape each engine really uses;
• a public leaderboard.

License

Apache-2.0