nosmpl 0.0.2

NoSMPL: Optimized common used SMPL operation.

NoSMPL

An enchanced and accelerated SMPL operation which commonly used in 3D human mesh generation. It takes a poses, shapes, cam_trans as inputs, outputs a high-dimensional 3D mesh verts.

However, SMPL codes and models are so messy out there, they have a lot of codes do calculation, some of them can not be easily deployed or accerlarated. So we have nosmpl here, it provides:

• build on smplx, but with onnx support;
• can be inference via onnx;
• we also demantrated some using scenarios infer with nosmpl but without any model, only onnx.

This packages provides:

• Highly optimized pytorch acceleration with FP16 infer enabled;
• Supported ONNX export and infer via ort, so that it might able used into TensorRT or OpenVINO on cpu;
• Support STAR, next generation of SMPL.
• Provide commonly used geoemtry built-in support without torchgeometry or kornia.

STAR model download from: https://star.is.tue.mpg.de/downloads

Updates

• 2022.03.03: I add some box_transform code into nosmpl, no we can get box_scale info when recover cropped img predicted 3d vertices back to original image. This is helpful when you project 3d vertices back to original image when using realrender. the usage like:

  from nosmpl.box_trans import get_box_scale_info, convert_vertices_to_ori_img
  box_scale_o2n, box_topleft, _ = get_box_scale_info(img, bboxes)
  frame_verts = convert_vertices_to_ori_img(
            frame_verts, s, t, box_scale_o2n, box_topleft
        )
  

• 2022.03.05: More to go.

Examples

an example to call nosmlp:

from nosmpl.smpl import SMPL

smpl = SMPL(smplModelPath, extra_regressor='extra_data/body_module/data_from_spin/J_regressor_extra.npy').to(device)

# get your betas and rotmat
pred_vertices, pred_joints_3d, faces = smpl(
                    pred_betas, pred_rotmat
                ) 

# note that we returned faces in SMPL model, you can use for visualization
# joints3d will add extra joints if you have extra_regressor like in SPIN or VIBE

The output shape of onnx model like:

                    basicModel_neutral_lbs_10_207_0_v1.0.0.onnx Detail
╭───────────────┬────────────────────────────┬──────────────────────────┬────────────────╮
│ Name          │ Shape                      │ Input/Output             │ Dtype          │
├───────────────┼────────────────────────────┼──────────────────────────┼────────────────┤
│ 0             │ [1, 10]                    │ input                    │ float32        │
│ 1             │ [1, 24, 3, 3]              │ input                    │ float32        │
│ verts         │ [-1, -1, -1]               │ output                   │ float32        │
│ joints        │ [-1, -1, -1]               │ output                   │ float32        │
│ faces         │ [13776, 3]                 │ output                   │ int32          │
╰───────────────┴────────────────────────────┴──────────────────────────┴────────────────╯
                             Table generated by onnxexplorer

Results

Some pipelines build with nosmpl support.

Copyrights

Copyrights belongs to Copyright (C) 2020 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG) and Lucas Jin