3D simulation environment
Project description
ThreeDWorld (TDW)
ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. With TDW, users can simulate high-fidelity sensory data and physical interactions between mobile agents and objects in a wide variety of rich 3D environments.
General guide to TDW
Setup
Core Concepts
- The controller
- Auto-launching the TDW build
- Commands
- Design philosophy of TDW
- Scenes
- Avatars and cameras
- Add-ons and the
ThirdPersonCamera
- Objects
- Output data
- Images
Troubleshooting
- How to report an issue
- Common errors
- Performance optimizations
- Good coding practices
- The
Logger
add-on
Tutorials
1. Scene Setup
1.1 Scene Setup (High-Level APIs)
- Overview
- Procedural generation (the
ProcGenKitchen
add-on) - Regions, interior regions, and rooms
- Procedural object arrangements
- Scripted object placement (floorplan layouts)
- Reset a scene
High-level APs: Floorplan and ProcGenKitchen
1.2 Scene Setup (Low-Level APIs)
- Overview
- Units and data formats
Bounds
output data- Visual materials, textures, and colors
- ProcGenRoom
2. 3D Model Libraries
3. Semantic States
- Overview
- Line of sight (
Raycast
output data) - Proximity to region (
Overlap
output data) - Proximity to other objects (the
TriggerCollisionManager
add-on) - Containment (the
ContainerManager
add-on) - Composite objects (objects with affordances)
- Grasped objects
4. Visual Perception
- Overview
- Instance ID segmentation colors (
_id
pass) - Semantic category segmentation colors (
_category
pass) - Depth maps (
_depth
and_depth_simple
passes) - Motion perception (
_flow
pass) - Other image passes (
_mask
,_normals
, and_albedo
passes) Occlusion
output data
5. Camera Controls
6. Photorealism
- Overview
- Lighting (HDRI skyboxes)
- Post-processing
- Interior lighting (the
InteriorSceneLighting
add-on) - Depth of field
High-level API: tdw_image_dataset
7. Physics
7.1 Physics (PhysX)
- PhysX
- Object physics parameters
Rigidbodies
output dataCollision
output data- Apply forces to objects
- Skip physics frames
- Disable physics
High-level API: tdw_physics
7.2 Physics (Obi)
- Obi
- Fluids
- Cloth
ObiParticles
output data- Colliders and collision materials
- Solvers
- Obi and robots
- Obi and Flex
7.3 Physics (Flex)
- Flex
- Solid and soft actors
- Cloth actors
- Fluid and source actors
- Move, rotate, and scale Flex objects
FlexParticles
output data- Apply forces to Flex objects
- Reset a Flex scene
- Other Flex commands
High-level API: tdw_physics
8. Audio
- Overview
- Initialize audio and play .wav files
- Resonance Audio
PyImpact
(dynamic impact sounds)- Recording audio
PyImpact
(advanced API)- Audio perception
9. Video Recording
10. Agents
10.1 Robots
- Overview
- The
Robot
add-on - Robot arm add-ons
- Robot collision detection
- Select a robot
- Add your own robots to TDW
- Robotics API (low-level)
- Add a camera to a robot
10.2 Magnebots
10.3 Virtual Reality (VR)
10.4 Keyboard and Mouse
10.5 Embodied avatars
11. Multi-agent simulations
12. Navigation
13. Non-physics objects
- Overview
- Position markers
- Line renderers
- Textured quads
- User Interface (the
UI
add-on) - Compass rose
- Non-physics humanoids
14. Misc. remote server topics
15. Misc. other topics
API Documentation
Command API
tdw
module API
tdw
- AssetBundleCreator
- AssetBundleCreatorBase
- AudioConstants
- AudioUtils
- CardinalDirection
- Controller
- IntPair
- OrdinalDirection
- QuaternionUtils
- RemoteBuildLauncher
- RobotCreator
- TDWUtils
tdw.add_ons
- AddOn
- AudioInitializer
- AudioInitializerBase
- AvatarBody
- Benchmark
- CinematicCamera
- CollisionManager
- CompositeObjectManager
- ContainerManager
- EmbodiedAvatar
- FirstPersonAvatar
- Floorplan
- ImageCapture
- InteriorSceneLighting
- Keyboard
- Logger
- ModelVerifier
- Mouse
- Obi
- ObjectManager
- OccupancyMap
- OculusTouch
- PhysicsAudioRecorder
- ProcGenKitchen
- PyImpact
- ResonanceAudioInitializer
- Robot
- RobotArm
- RobotBase
- StepPhysics
- ThirdPersonCamera
- ThirdPersonCameraBase
- TriggerCollisionManager
- UI
- VR
tdw.collision_data
tdw.container_data
tdw.flex_data
tdw.librarian
- HdriSkyboxLibrarian
- HumanoidAnimationLibrarian
- HumanoidLibrarian
- MaterialLibrarian
- ModelLibrarian
- RobotLibrarian
- SceneLibrarian
tdw.model_tests
tdw.obi_data
tdw.obi_data.cloth
tdw.obi_data.collision_materials
tdw.obi_data.fluids
- CubeEmitter
- DiskEmitter
- EdgeEmitter
- EmitterSamplingMethod
- EmitterShape
- Fluid
- FluidBase
- GranularFluid
- SphereEmitter
tdw.object_data
tdw.object_data.composite_object
tdw.object_data.composite_object.sub_object
- HingeDynamic
- HingeStatic
- HingeStaticBase
- LightDynamic
- LightStatic
- MotorStatic
- NonMachineStatic
- PrismaticJointStatic
- SpringStatic
- SubObjectDynamic
- SubObjectStatic
tdw.physics_audio
- AudioMaterial
- Base64Sound
- CollisionAudioEvent
- CollisionAudioInfo
- CollisionAudioType
- Modes
- ObjectAudioStatic
- ScrapeMaterial
- ScrapeModel
- ScrapeSubObject
tdw.proc_gen.arrangements
- Arrangement
- ArrangementAlongWall
- ArrangementWithRootObject
- Basket
- CupAndCoaster
- Dishwasher
- KitchenCabinet
- KitchenCounter
- KitchenCounterTop
- KitchenTable
- Microwave
- Painting
- Plate
- Radiator
- Refrigerator
- Shelf
- SideTable
- Sink
- StackOfPlates
- Stool
- Stove
- Suitcase
- TableAndChairs
- TableSetting
- Void
- WallCabinet
tdw.proc_gen.arrangements.cabinetry
tdw.release
tdw.robot_data
tdw.scene_data
tdw.vr_data
Performance benchmarks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.