Skip to main content

3D face reconstruction from a single image

Project description

Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation - Official PyTorch Implementation

Binder PyPI version License: MIT

[Arxiv] [Video]

Evaluation code for Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation. Finally ported to PyTorch!

Recent Updates

2020.10.27: Added STL support

2020.05.07: Added a wheel package!

2020.05.06: Added myBinder version for quick testing of the model

2020.04.30: Initial pyTorch release

What's in this release?

The original pix2vertex repo was composed of three parts

  • A network to perform the image to depth + correspondence maps trained on synthetic facial data
  • A non-rigid ICP scheme for converting the output maps to a full 3D Mesh
  • A shape-from-shading scheme for adding fine mesoscopic details

This repo currently contains our image-to-image network with weights and model to PyTorch and a simple python postprocessing scheme.

  • The released network was trained on a combination of synthetic images and unlabeled real images for some extra robustness :)

Installation

Installation from PyPi

    $ pip install pix2vertex

Installation from source

    $ git clone https://github.com/eladrich/pix2vertex.pytorch.git
    $ cd pix2vertex.pytorch
    $ python setup.py install

Usage

The quickest way to try p2v is using the reconstruct method over an input image, followed by visualization or STL creation.

 import pix2vertex as p2v
 from imageio import imread

 image = imread(<some image file>)
 result, crop = p2v.reconstruct(image)

# Interactive visualization in a notebook
 p2v.vis_depth_interactive(result['Z_surface'])

# Static visualization using matplotlib
p2v.vis_depth_matplotlib(crop, result['Z_surface'])

# Export to STL
p2v.save2stl(result['Z_surface'], 'res.stl')

For a more complete example see the reconstruct_pipeline notebook. You can give it a try without any installations using our binder port.

Pretrained Model

Models can be downloaded from these links:

If no model path is specified the package automagically downloads the required models.

TODOs

  • Port Torch model to PyTorch
  • Release an inference notebook (using K3D)
  • Add requirements
  • Pack as wheel
  • Ported to MyBinder
  • Add a simple method to export a stl file for printing
  • Port the Shape-from-Shading method used in our matlab paper
  • Write a short blog about the revised training scheme

Citation

If you use this code for your research, please cite our paper Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation:

@article{sela2017unrestricted,
  title={Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation},
  author={Sela, Matan and Richardson, Elad and Kimmel, Ron},
  journal={arxiv},
  year={2017}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pix2vertex-1.0.4.tar.gz (14.0 kB view details)

Uploaded Source

Built Distribution

pix2vertex-1.0.4-py3-none-any.whl (13.5 kB view details)

Uploaded Python 3

File details

Details for the file pix2vertex-1.0.4.tar.gz.

File metadata

  • Download URL: pix2vertex-1.0.4.tar.gz
  • Upload date:
  • Size: 14.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.22.0 setuptools/45.2.0.post20200210 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.6

File hashes

Hashes for pix2vertex-1.0.4.tar.gz
Algorithm Hash digest
SHA256 12ff6fe3bf895fa77ead8a40b3c13cb5444e703f4a461f8607d00a029ba73fe3
MD5 b540e008b0f109b9a38c35dcfcc1a2ca
BLAKE2b-256 6b98e6f7880d3d7e76fe3ce72d69d206ffbc50cb72a1c81e6d121c2b4d763dbf

See more details on using hashes here.

File details

Details for the file pix2vertex-1.0.4-py3-none-any.whl.

File metadata

  • Download URL: pix2vertex-1.0.4-py3-none-any.whl
  • Upload date:
  • Size: 13.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.22.0 setuptools/45.2.0.post20200210 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.6

File hashes

Hashes for pix2vertex-1.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 06ff439c9ee21b651fad6cce0b2fe49cba1efc4cbc0c90625748e52115f82567
MD5 0b85ff2f95e1b2e7bf5bafe9818b397d
BLAKE2b-256 4a45ca11e0c3f84c53fc5a01fe3feceda1b75917de3146f4051a0089abc3fabe

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page