face-landmarker-3d 1.0.1

3D Face Landmarker - MediaPipe-based face landmark detection

MVMP: 3D Multi-View MediaPipe

Description

MVMP (Multi-View MediaPipe) is a lightweight software designed for the 3D localization of facial landmarks starting from a static textured mesh. It generates multiple orthographic projections of the face mesh, estimates 2D landmarks using MediaPipe, and backprojects them into 3D space through a consensus-based method. The result is a robust prediction of 478 facial landmarks directly aligned with the 3D mesh geometry.

This method enables landmark estimation even without original scanning data or active vision systems, offering an efficient and flexible approach for 3D face modeling, recognition, and analysis.

Table of Contents

• Installation
• Usage
• Contributing
• License
• Contact

Installation

To install and set up the project, follow these steps:

1. Ensure you have Python 3.11 installed.

2. Create and activate a virtual environment (optional but recommended):

   python3.11 -m venv myenv
   source myenv/bin/activate   # On Linux/macOS
   myenv\Scripts\activate      # On Windows
   

3. Install the required packages:

   pip install -r requirements.txt
   

4. Download the MediaPipe Face Landmarker model:

   wget https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/latest/face_landmarker.task
   

Usage

The main entry point for the program is predict.py.

You can run it with:

python predict.py --help

It accepts the following optional arguments:

optional arguments:
  -h, --help            Show this help message and exit.
  -f, --folder          Indicates that the input path is a root folder that will be accessed recursively to find .obj files.
  -p PROJECTIONS_NUMBER, --projections-number PROJECTIONS_NUMBER
                        Number of projections to be calculated. Default is 500.
  -o OUTPUT_PATH, --output-path OUTPUT_PATH
                        Optional. If set, results will be saved to this path. Otherwise, they will be saved inside the .obj file folder.
  -r, --render          If set, renders the results in an external window.

Output

For each processed mesh, a <mesh>_landmarks.json file will be generated inside the corresponding mesh's folder (or the specified output folder if --output-path is used).

The JSON file contains:

• The normalized coordinates of the estimated 478 landmarks.
• The index of the closest mesh vertex for each landmark.

Results

Contributing

We welcome contributions! To contribute:

1. Fork the repository and create a feature branch.
2. Make your changes with clear and descriptive commit messages.
3. Push the branch to your fork.
4. Open a pull request describing your changes.

License

This project is licensed under the MIT License.

Contact

If you have any questions or suggestions, feel free to get in touch!