Refined-Augment 0.1.5

This package allows you to detect faces in real-time using a webcam and overlay an AR object above the detected face.

AR Face Overlay Package

This package allows you to detect faces in real-time using a webcam and overlay an image (sticker) above the detected face. It includes built-in system checks to ensure your hardware can run the processing smoothly.

Prerequisites: The package requires Python 3.7 or higher and the following libraries:

• opencv-python
• numpy
• psutil

they will be Installed automatically using pip:

pip install opencv-python numpy psutil

Usage:

You can start the application by importing the package in your main script (test.py for example).

import cv2
import numpy as np
from Refined_Augment import Refined_Augment

ar = Refined_Augment()

cap = cv2.VideoCapture(0) 

while True:

    ret, frame = cap.read()

    imgAug = ar.overlay(frame, "AR_photo.png",
                        use_haar=True,
                        # manual_faces=None,
                        show_bounding_box=True)

    cv2.imshow('Program', imgAug)

# Mediapipe hand support example
# If you want the overlay to follow your hand, call overlay with target='hand'.
# Use `use_mediapipe=True` to let the library detect hands automatically.
# The overlay size will adjust based on hand spread.
# imgAug = ar.overlay(frame, "AR_photo.png", target='hand', use_mediapipe=True, show_bounding_box=True, position='infront')

# 3D OBJ model overlay example
# You can also overlay 3D models from .obj files.
# imgAug = ar.overlay(frame, "model.obj", target='face', position='infront')

    if not ret:
        print('failed to grab frame')
        break
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

and to see a step by step component , the output of each step, run this in your cell or script:

import cv2
import numpy as np

cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        print('failed to grab frame')
        break

    # Your AR processing
    imgAug = ar.overlay(frame, "AR_photo.png",
                        use_haar=True,
                        show_bounding_box=True)
    
    overlay = ar.get_overlay_image()
    warped_image = ar.get_warped_image()
    masknew, maskinv = ar.get_mask()


    masknew_3ch = cv2.cvtColor(masknew, cv2.COLOR_GRAY2BGR)
    maskinv_3ch = cv2.cvtColor(maskinv, cv2.COLOR_GRAY2BGR)

    # Ensure all images are the same size (using the original frame size as the standard)
    h, w = frame.shape[:2]
    images_to_stack = [frame, imgAug, overlay, warped_image, masknew_3ch, maskinv_3ch]
    
    resized_images = [cv2.resize(img, (w, h)) for img in images_to_stack]

    # --- Stacking ---
    # Horizontal stack (side-by-side)
    h_stack = cv2.hconcat(resized_images)
    
    
    # Note: If the stack is too wide for your screen, consider a 2x3 grid
    top_row = cv2.hconcat(resized_images[0:3])
    bottom_row = cv2.hconcat(resized_images[3:6])
    combined_view = cv2.vconcat([top_row, bottom_row])

    cv2.imshow('AR Pipeline Stages', combined_view)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

How It Works :

• System Check: The program verifies if you have at least 2GB of RAM and 2 CPU cores to prevent lag.

• Resource Management: It automatically searches for the haarcascade_frontalface_default.xml file. If the file is not found locally or in the OpenCV system folder, it downloads it from the official repository.

• Perspective Warping: The program uses a homography matrix to scale and position the overlay image so it follows the movement of the face.

File Descriptions :

Diagnostics.py: Contains functions to check RAM, CPU, and camera availability. It also handles the path resolution for the Haar Cascade XML file.

Engine.py: Contains the main loop that processes video frames, detects faces, and applies the image overlay logic.

init.py: Acts as the package interface, coordinating the diagnostics and the engine.

Controls :

'q': Press the 'q' key on your keyboard to stop the video feed and close the application.

Troubleshooting Image Load Error: Ensure the image path provided in ar.start() is correct relative to where you are running the script.

Camera Error: If the camera access fails, check if another application is using the webcam.

Persistence Error: This occurs if the XML file is corrupted or missing. The program will attempt to re-download it if you delete the existing XML file in the directory.