Sinapsis Retina Face TRT

Templates for real-time facial recognition with RetinaFace and DeepFace.

🐍 Installation • 🚀 Features • 📚 Example usage • 🌐 Webapp 📙 Documentation • 🔍 License

The sinapsis-retina-face-trt module provides templates for real-time facial recognition with RetinaFace and DeepFace, enabling efficient and accurate inference.

🐍 Installation

[!NOTE] CUDA-based templates in sinapsis-retina-face-trt require NVIDIA driver version to be 550 or higher.

Install using your package manager of choice. We encourage the use of uv

uv pip install sinapsis-retina-face-trt

or wiht raw pip

pip install sinapsis-retina-face-trt 

[!IMPORTANT] To enable tensorflow with CUDA support please install tensorflow as follows:

uv pip install tensorflow[and-cuda]==2.18.0

or

pip install tensorflow[and-cuda]==2.18.0

[!IMPORTANT] Templates in sinapsis-retina-face-trt package may require extra dependencies. For development, we recommend installing the package with all the optional dependencies:

uv pip install sinapsis-retina-face-trt[all] --extra-index-url https://pypi.sinapsis.tech

or

pip install sinapsis-retina-face-trt[all] --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

The Sinapsis Retina Face TRT module provides multiple templates for real-time facial recognition, leveraging TensorRT optimization and DeepFace embedding search.

• RetinaFacePytorch. Runs face detection using RetinaFace implemented in PyTorch.
• RetinaFacePytorchTRT. A TensorRT-optimized version of RetinaFacePytorch for faster inference.
• RetinaFacePytorchTRTTorchOnly. A Torch-TensorRT optimized version of RetinaFace, focusing solely on Torch-TRT acceleration.
• PytorchEmbeddingSearch. Performs similarity search over a gallery of embeddings.
• PytorchEmbeddingExtractor. A base template for extracting embeddings from face images.
• Facenet512EmbeddingExtractorTRT. Uses TensorRT for fast embedding extraction based on Facenet512.
• Facenet512EmbeddingExtractorTRTDev. An alternative version of Facenet512EmbeddingExtractorTRT that converts the model at runtime.
• FaceVerificationFromGallery. Perform face verification by direct comparison between predicted face embeddings and face embeddings stored in a gallery file.

📚 Example usage

The following example demonstrates how to use the RetinaFacePytorchTRT template for real-time facial detection.

This configuration defines an agent and a sequence of templates to run real-time facial recognition with RetinaFace.

1. Image Loading (FolderImageDatasetCV2): Loads images from the specified directory (data_dir).
2. Face Detection (RetinaFacePytorchTRT): Runs inference using RetinaFace, applying a confidence threshold, model configuration, and pretrained weights.
3. Bounding Box Drawing (BBoxDrawer): Overlays bounding boxes on detected faces.
4. Saving Results (ImageSaver): Saves the processed images to the defined output directory.

agent:
  name: face_detection
  description: > 
    Agent to perform face detection by employing an accelerated TRT version of the RetinaFace model.

templates:
- template_name: InputTemplate-1
  class_name: InputTemplate
  attributes: {}

- template_name: FolderImageDatasetCV2-1
  class_name: FolderImageDatasetCV2
  template_input: InputTemplate-1
  attributes:
    data_dir: /opt/app/datasets/vision/detection/lfw
    load_on_init : true
    samples_to_load : 1
    batch_size : 10

- template_name: RetinaFacePytorch-1
  class_name: RetinaFacePytorchTRT
  template_input: FolderImageDatasetCV2-1
  attributes:
    return_key_points: true
    confidence_threshold: 0.3
    local_model_path: "/root/.cache/torch/hub/checkpoints/resnet50_2020-07-20.engine"

- template_name: BBoxDrawer
  class_name: BBoxDrawer
  template_input: RetinaFacePytorch-1
  attributes:
    draw_boxes: true
    draw_key_points: true
    randomized_color: false

- template_name: ImageSaver
  class_name: ImageSaver
  template_input: BBoxDrawer
  attributes:
    save_dir: "examples/inference_results/"
    root_dir: ""
    extension: jpg
    save_full_image: true
    save_bbox_crops: false

To run the agent, you should run:

sinapsis run /path/to/sinapsis-retina-face-trt/src/sinapsis_retina_face_trt/configs/face_recognition.yml

🌐 Webapp

The webapps included in this repo provide interactive interfaces to showcase real-time facial recognition and face verification mode capabilities.

[!IMPORTANT] To run the apps, you first need to clone this repository:

git clone https://github.com/sinapsis-ai/sinapsis-retina-face-trt.git

[!NOTE] The face recognition app requires a dataset of face images organized into folders, where each folder is named after the individual whose face images it contains. Example dataset structure:

.
└── gallery/
    ├── person_1/
    │   ├── image_1
    │   ├── image_2
    │   ├── image_3
    │   └── image_4
    ├── person_2/
    │   ├── image_1
    │   ├── image_2
    │   ├── image_3
    │   └── image_4
    └── person_3/
        ├── image_1
        ├── image_2
        └── image_3

We have created a small version of the lfw dataset in the following link

[!NOTE] The face verification app don't require you to build a dataset. For demo purposes, the app is designed to perform face validation by using only one image as reference which should be provided through the app interface.

[!WARNING] If you have cached versions of the retinaface or Facenet models, please remove them before running the app. To remove cached versions, use (might need root permissions, in which case use sudo)

rm -rf ~/.cache/torch/hub/checkpoints/* && rm -rf ~/.cache/sinapsis/.deepface/weights/*

[!NOTE] If you'd like to enable external app sharing in Gradio use: export GRADIO_SHARE_APP=True

docker compose -f docker/compose.yaml build

export GALLERY_ROOT_DIR=/path/to/dataset/

docker compose -f docker/compose_apps.yaml up sinapsis-face-recognition-gradio -d

docker compose -f docker/compose_apps.yaml up sinapsis-verification-mode-gradio -d

docker logs -f sinapsis-face-recognition-gradio

docker logs -f sinapsis-verification-mode-gradio 

Running on local URL:  http://127.0.0.1:7860

docker compose -f docker/compose_apps.yaml down

uv sync --frozen

uv pip install sinapsis-retina-face-trt[all] --extra-index-url https://pypi.sinapsis.tech

uv pip install tensorflow[and-cuda]==2.18.0

uv run webapps/face_recognition_demo.py

uv run webapps/verification_mode_demo.py

Running on local URL:  http://127.0.0.1:7860

📙 Documentation

Documentation is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.