NoSMPL: Optimized common used SMPL operation.
Project description
NoSMPL
An enchanced and accelerated SMPL operation which commonly used in 3D human mesh generation. It takes a poses, shapes, cam_trans as inputs, outputs a high-dimensional 3D mesh verts.
However, SMPL codes and models are so messy out there, they have a lot of codes do calculation, some of them can not be easily deployed or accerlarated. So we have nosmpl
here, it provides:
- build on smplx, but with onnx support;
- can be inference via onnx;
- we also demantrated some using scenarios infer with
nosmpl
but without any model, only onnx.
This packages provides:
- Highly optimized pytorch acceleration with FP16 infer enabled;
- Supported ONNX export and infer via ort, so that it might able used into TensorRT or OpenVINO on cpu;
- Support STAR, next generation of SMPL.
- Provide commonly used geoemtry built-in support without torchgeometry or kornia.
STAR model download from: https://star.is.tue.mpg.de/downloads
Updates
2022.05.16
: Now addedhuman_prior
insidenosmpl
, you don't need install that lib anymore, or install torchgeometry either:from nosmpl.vpose.tools.model_loader import load_vposer self.vposer, _ = load_vposer(VPOSER_PATH, vp_model="snapshot")
then you can load vpose to use.2022.05.10
: Add BHV reader, you can now read and write bvh file:from nosmpl.parsers import bvh_io import sys animation = bvh_io.load(sys.argv[1]) print(animation.names) print(animation.frametime) print(animation.parent) print(animation.offsets) print(animation.shape)
2022.05.07
: Added a visualization for Human36m GT, you can using like this to visualize h36m data now:
Just send a h36m txt annotation file, and you can see the animation result. Also, you can usingimport nosmpl.datasets.h36m_data_utils as data_utils from nosmpl.datasets.h36m_vis import h36m_vis_on_gt_file import sys if __name__ == "__main__": h36m_vis_on_gt_file(sys.argv[1])
from nosmpl.datasets.h36m_vis import h36m_load_gt_3d_data
to load 3d data in 3D space.2022.03.03
: I add somebox_transform
code intonosmpl
, no we can get box_scale info when recover cropped img predicted 3d vertices back to original image. This is helpful when you project 3d vertices back to original image when usingrealrender
. the usage like:from nosmpl.box_trans import get_box_scale_info, convert_vertices_to_ori_img box_scale_o2n, box_topleft, _ = get_box_scale_info(img, bboxes) frame_verts = convert_vertices_to_ori_img( frame_verts, s, t, box_scale_o2n, box_topleft )
2022.03.05
: More to go.
Examples
an example to call nosmlp
:
from nosmpl.smpl import SMPL
smpl = SMPL(smplModelPath, extra_regressor='extra_data/body_module/data_from_spin/J_regressor_extra.npy').to(device)
# get your betas and rotmat
pred_vertices, pred_joints_3d, faces = smpl(
pred_betas, pred_rotmat
)
# note that we returned faces in SMPL model, you can use for visualization
# joints3d will add extra joints if you have extra_regressor like in SPIN or VIBE
The output shape of onnx model like:
basicModel_neutral_lbs_10_207_0_v1.0.0.onnx Detail
╭───────────────┬────────────────────────────┬──────────────────────────┬────────────────╮
│ Name │ Shape │ Input/Output │ Dtype │
├───────────────┼────────────────────────────┼──────────────────────────┼────────────────┤
│ 0 │ [1, 10] │ input │ float32 │
│ 1 │ [1, 24, 3, 3] │ input │ float32 │
│ verts │ [-1, -1, -1] │ output │ float32 │
│ joints │ [-1, -1, -1] │ output │ float32 │
│ faces │ [13776, 3] │ output │ int32 │
╰───────────────┴────────────────────────────┴──────────────────────────┴────────────────╯
Table generated by onnxexplorer
Results
Some pipelines build with nosmpl
support.
Copyrights
Copyrights belongs to Copyright (C) 2020 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG) and Lucas Jin
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.