sinapsis-retina-face-trt 0.1.14

trt version of retina face and facenet

Sinapsis Retina Face TRT

Templates for real-time facial recognition with RetinaFace and DeepFace.

🐍 Installation • 🚀 Features • 📚 Example usage • 🌐 Webapp 📙 Documentation • 🔍 License

The sinapsis-retina-face-trt module provides templates for real-time facial recognition with RetinaFace and DeepFace, enabling efficient and accurate inference.

🐍 Installation

[!NOTE] CUDA-based templates in sinapsis-retina-face-trt require NVIDIA driver version to be 550 or higher.

Install using your package manager of choice. We encourage the use of uv

uv pip install sinapsis-retina-face-trt

or wiht raw pip

pip install sinapsis-retina-face-trt 

[!IMPORTANT] To enable tensorflow with CUDA support please install tensorflow as follows:

uv pip install tensorflow[and-cuda]==2.18.0

or

pip install tensorflow[and-cuda]==2.18.0

[!IMPORTANT] Templates in sinapsis-retina-face-trt package may require extra dependencies. For development, we recommend installing the package with all the optional dependencies:

uv pip install sinapsis-retina-face-trt[all] --extra-index-url https://pypi.sinapsis.tech

or

pip install sinapsis-retina-face-trt[all] --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

The Sinapsis Retina Face TRT module provides multiple templates for real-time facial recognition, leveraging TensorRT optimization and DeepFace embedding search.

• RetinaFacePytorch. Runs face detection using RetinaFace implemented in PyTorch.
• RetinaFacePytorchTRT. A TensorRT-optimized version of RetinaFacePytorch for faster inference.
• RetinaFacePytorchTRTTorchOnly. A Torch-TensorRT optimized version of RetinaFace, focusing solely on Torch-TRT acceleration.
• PytorchEmbeddingSearch. Performs similarity search over a gallery of embeddings.
• PytorchEmbeddingExtractor. A base template for extracting embeddings from face images.
• Facenet512EmbeddingExtractorTRT. Uses TensorRT for fast embedding extraction based on Facenet512.
• Facenet512EmbeddingExtractorTRTDev. An alternative version of Facenet512EmbeddingExtractorTRT that converts the model at runtime.
• FaceVerificationFromGallery. Perform face verification by direct comparison between predicted face embeddings and face embeddings stored in a gallery file.

📚 Example usage

The following example demonstrates how to use the RetinaFacePytorchTRT template for real-time facial detection.

This configuration defines an agent and a sequence of templates to run real-time facial recognition with RetinaFace.

1. Image Loading (FolderImageDatasetCV2): Loads images from the specified directory (data_dir).
2. Face Detection (RetinaFacePytorchTRT): Runs inference using RetinaFace, applying a confidence threshold, model configuration, and pretrained weights.
3. Bounding Box Drawing (BBoxDrawer): Overlays bounding boxes on detected faces.
4. Saving Results (ImageSaver): Saves the processed images to the defined output directory.

Config file

agent:
  name: face_detection
  description: > 
    Agent to perform face detection by employing an accelerated TRT version of the RetinaFace model.

templates:
- template_name: InputTemplate-1
  class_name: InputTemplate
  attributes: {}

- template_name: FolderImageDatasetCV2-1
  class_name: FolderImageDatasetCV2
  template_input: InputTemplate-1
  attributes:
    data_dir: /opt/app/datasets/vision/detection/lfw
    load_on_init : true
    samples_to_load : 1
    batch_size : 10

- template_name: RetinaFacePytorch-1
  class_name: RetinaFacePytorchTRT
  template_input: FolderImageDatasetCV2-1
  attributes:
    return_key_points: true
    confidence_threshold: 0.3
    local_model_path: "/root/.cache/torch/hub/checkpoints/resnet50_2020-07-20.engine"

- template_name: BBoxDrawer
  class_name: BBoxDrawer
  template_input: RetinaFacePytorch-1
  attributes:
    draw_boxes: true
    draw_key_points: true
    randomized_color: false

- template_name: ImageSaver
  class_name: ImageSaver
  template_input: BBoxDrawer
  attributes:
    save_dir: "examples/inference_results/"
    root_dir: ""
    extension: jpg
    save_full_image: true
    save_bbox_crops: false

To run the agent, you should run:

sinapsis run /path/to/sinapsis-retina-face-trt/src/sinapsis_retina_face_trt/configs/face_recognition.yml

🌐 Webapp

The webapps included in this repo provide interactive interfaces to showcase real-time facial recognition and face verification mode capabilities.

[!IMPORTANT] To run the apps, you first need to clone this repository:

git clone https://github.com/sinapsis-ai/sinapsis-retina-face-trt.git

[!NOTE] The face recognition app requires a dataset of face images organized into folders, where each folder is named after the individual whose face images it contains. Example dataset structure:

.
└── gallery/
    ├── person_1/
    │   ├── image_1
    │   ├── image_2
    │   ├── image_3
    │   └── image_4
    ├── person_2/
    │   ├── image_1
    │   ├── image_2
    │   ├── image_3
    │   └── image_4
    └── person_3/
        ├── image_1
        ├── image_2
        └── image_3

We have created a small version of the lfw dataset in the following link

[!NOTE] The face verification app don't require you to build a dataset. For demo purposes, the app is designed to perform face validation by using only one image as reference which should be provided through the app interface.

[!WARNING] If you have cached versions of the retinaface or Facenet models, please remove them before running the app. To remove cached versions, use (might need root permissions, in which case use sudo)

rm -rf ~/.cache/torch/hub/checkpoints/* && rm -rf ~/.cache/sinapsis/.deepface/weights/*

[!NOTE] If you'd like to enable external app sharing in Gradio use: export GRADIO_SHARE_APP=True

🐳 Docker

1. Build the sinapsis-retina-face-trt image:

docker compose -f docker/compose.yaml build

2. Start the container:

For face recognition app, export the variable with the path to your gallery folder

export GALLERY_ROOT_DIR=/path/to/dataset/

and initialize app

docker compose -f docker/compose_apps.yaml up sinapsis-face-recognition-gradio -d

For face verification app

docker compose -f docker/compose_apps.yaml up sinapsis-verification-mode-gradio -d

3. Check the status:

For face recognition app

docker logs -f sinapsis-face-recognition-gradio

For face verification app

docker logs -f sinapsis-verification-mode-gradio 

4. The logs will display the URL to access the webapp:

Running on local URL:  http://127.0.0.1:7860

5. To stop the app:

docker compose -f docker/compose_apps.yaml down

📦 UV

1. Create the virtual environment and sync the dependencies:

uv sync --frozen

2. Install the sinapsis-retina-face-trt package with all its dependencies:

uv pip install sinapsis-retina-face-trt[all] --extra-index-url https://pypi.sinapsis.tech

3. Install tensorflow with cuda support:

uv pip install tensorflow[and-cuda]==2.18.0

4. Run the webapp.

For face recognition app:

Update the following attributes in the face_recognition config file:

• local_model_path in the RetinaFacePytorch-1 template, to point to the torch hub cache local folder.
• image_root_dir in the PytorchEmbeddingSearch-1 template, to point to your local gallery folder.

then run:

uv run webapps/face_recognition_demo.py

For face verification app:

Update the local_model_path attributes of the RetinaFacePytorch-1 template in the face_verification config file to point to the torch hub cache local folder:

then run:

uv run webapps/verification_mode_demo.py

5. The terminal will display the URL to access the webapp:

Running on local URL:  http://127.0.0.1:7860

NOTE: The URL may vary; check the terminal output for the correct address.

📙 Documentation

Documentation is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.