Refined-Augment 0.1.7

This package allows you to detect faces in real-time using a webcam and overlay an AR object above the detected face.

Refined Augment

Refined Augment is a powerful Python package designed for enhancing and augmenting data using Artificial Intelligence. It specializes in Computer Vision tasks, allowing users to seamlessly overlay 2D images and 3D OBJ models onto faces and hands in real-time.

---

Features

• Dual Target Detection: Support for both Face and Hand landmarks.
• 3D Model Support: Render .obj files directly onto detected targets with automatic scaling.
• 2D Overlay Support: Perspective warping for .png, .jpg, and web-hosted images.
• Flexible Positioning: Place overlays above, below, left, right, or infront of the target.
• Multiple Detection Backends: Integration with Haar Cascades and MediaPipe.

---

Installation

Ensure you have the required dependencies installed:

pip install opencv-python numpy mediapipe scikit-image

---

Usage Samples

1. 3D Face Augmentation (Real-time)

This sample uses a 3D .obj model and overlays it directly "infront" of detected faces.

import cv2
from refined_augment import Refined_Augment

ar = Refined_Augment()
cap = cv2.VideoCapture(0) 

while True:
    ret, frame = cap.read()
    if not ret: break

    # Overlay a 3D wolf model on the face
    imgAug = ar.overlay(frame, "wolf.obj", target='face', position='infront')

    cv2.imshow('3D Face AR', imgAug)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

2. Hand Tracking with MediaPipe

Advanced hand augmentation using MediaPipe landmarks for precise object placement.

import cv2
import mediapipe as mp
from refined_augment import Refined_Augment

ar = Refined_Augment()
mp_hands = mp.solutions.hands
hands = mp_hands.Hands(
    static_image_mode=False,
    max_num_hands=1,
    min_detection_confidence=0.7,
    min_tracking_confidence=0.5
)

cap = cv2.VideoCapture(0)  

while True:
    ret, frame = cap.read()
    if not ret: break

    img_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    results = hands.process(img_rgb)
    imgAug = frame

    if results.multi_hand_landmarks:
        your_hand_landmarks = results.multi_hand_landmarks[0]
        
        # Apply the 3D model to the detected hand landmarks
        imgAug = ar.overlay(
            frame,
            "wolf.obj",
            target='hand',
            hand_landmarks=your_hand_landmarks,
            show_bounding_box=True,
            position='infront'
        )
    
    cv2.imshow('Hand AR', imgAug)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
hands.close()
cv2.destroyAllWindows()

3. Classic 2D Image Overlay

Overlay a standard image (like a mask or logo) onto faces using Haar Cascades.

import cv2
from refined_augment import Refined_Augment

ar = Refined_Augment()
cap = cv2.VideoCapture(0)  

while True:
    ret, frame = cap.read()
    if not ret: break

    # Overlay a 2D PNG on detected faces
    imgAug = ar.overlay(frame, "AR_photo.png",
                        use_haar=True,
                        manual_faces=None,
                        show_bounding_box=False)
    
    cv2.imshow('2D Overlay', imgAug)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

---

📖 API Documentation

Refined_Augment.overlay(...)

Parameter	Type	Description
image	ndarray	The input BGR frame from OpenCV.
overlay_path	str	Path to .obj or image file (Local or URL).
target	str	'face' or 'hand'.
position	str	'above', 'below', 'left', 'right', or 'infront'.
use_haar	bool	Uses Haar Cascades for face detection (Default: True).
hand_landmarks	list	Manual landmarks for hand positioning.
hand_scale_factor	float	Resize the object relative to the hand size.

---