aitviewer 1.3.0

Viewing and rendering of sequences of 3D data.

AITViewer

A set of tools to visualize and interact with sequences of 3D data with cross-platform support on Windows, Linux, and Mac OS X.

Features

• Easy to use Python interface.
• Load SMPL[-H | -X] / MANO / FLAME sequences and display them in an interactive viewer.
• Manually editable SMPL sequences.
• Render 3D data on top of images via weak-perspective or OpenCV camera models.
• Built-in extensible GUI (based on Dear ImGui).
• Prebuilt renderable primitives (cylinders, spheres, point clouds, etc).
• Render videos of the currently loaded sequences.
• Headless/Offscreen rendering.
• Support live data feeds and rendering (e.g., webcam).
• Modern OpenGL shader-based rendering pipeline for high performance (via ModernGL / ModernGL Window).

https://user-images.githubusercontent.com/5639197/188625409-1c86b12e-4e91-48ba-ab7e-b050508d586b.mp4

Installation

Basic Installation:

pip install aitviewer

Or install locally (if you need to extend or modify code)

git clone [e-mail]:eth-ait/aitviewer.git
cd aitviewer
pip install -e .

Note that this does not install the GPU-version of PyTorch automatically. If your environment already contains it, you should be good to go, otherwise install it manually.

Configuration

The viewer loads default configuration parameters from aitvconfig.yaml. There are two ways how to override these parameters:

• Create a file named aitvconfig.yaml and have the environment variable AITVRC point to it. Alternatively, you can point AITVRC to the directory containing aitvconfig.yaml.
• Create a file named aitvconfig.yaml in your current working directory, i.e. from where you launch your python program.

Note that the configuration files are loaded in this order, i.e. the config file in your working directory overrides all previous parameters.

The configuration management is using OmegaConf. You will probably want to override the following parameters at your convenience:

• datasets.amass: where AMASS is stored if you want to load AMASS sequences.
• smplx_models: where SMPLX models are stored, preprocessed as required by the smplx package.
• export_dir: where videos and other outputs are stored by default.

Quickstart

View an SMPL template

from aitviewer.renderables.smpl import SMPLSequence
from aitviewer.viewer import Viewer

smpl_template = SMPLSequence.t_pose()

# Display in viewer.
v = Viewer()
v.scene.add(smpl_template)
v.run()

Examples

Check out the examples for a few examples how to use the viewer:

load_3DPW.py: Loads an SMPL sequence from the 3DPW dataset and displays it in the viewer.

load_AMASS.py: Loads an SMPL sequence from the AMASS dataset and displays it in the viewer.

load_DIP.py: Loads an SMPL and IMU sequence taken from the TotalCapture dataset as used by DIP.

load_obj.py: Loads meshes from OBJ files.

load_ROMP.py: Loads the result of ROMP and overlays it on top of the input image with either a weak-perspective or an OpenCV camera.

load_template.py: Loads the template meshes of SMPL-H, MANO, and FLAME.

load_VIBE.py: Loads the result of VIBE and overlays it on top of the input image.

quick_start.py: The above quickstart example.

render_primitives.py: Renders a bunch of spheres and lines.

stream.py: Streams your webcam into the viewer.

vertex_clicking.py: An example how to subclass the basic Viewer class for custom interaction.

Projects using the AITViewer

The following projects have used the AITViewer:

• Dong et al., Shape-aware Multi-Person Pose Estimation from Multi-view Images, ICCV 2021
• Kaufmann et al., EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers, ICCV 2021
• Vechev et al., Computational Design of Kinesthetic Garments, Eurographics 2021
• Guo et al., Human Performance Capture from Monocular Video in the Wild, 3DV 2021
• Dong and Guo et al., PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence, CVPR 2022

Citation

If you use this software, please cite it as below.

@software{Kaufmann_Vechev_AITViewer_2022,
  author = {Kaufmann, Manuel and Vechev, Velko and Mylonopoulos, Dario},
  doi = {10.5281/zenodo.1234},
  month = {7},
  title = {{AITViewer}},
  url = {https://github.com/eth-ait/aitviewer},
  year = {2022}
}

Contact & Contributions

This software was developed by Manuel Kaufmann and Velko Vechev and Dario Mylonopoulos. For questions please create an issue. We welcome and encourage module and feature contributions from the community.