(Face Identification + Vision Events)

This project provides:

• Face identification pipelines (InceptionResnetV1 and ViT-based)
• Training/evaluation/tuning scripts
• A real-time Vision interface (interaction.vision.Vision) that emits user identity events

1) Setup (minimal)

Prerequisites

• Python 3.12
• Conda (recommended) or pip

Option A: Conda (recommended)

conda env create -f environment.yml
conda activate dissertation

Option B: pip

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Core commands

Run from repo root:

python core/models/inception_resnet.py
python core/models/vision_transformer.py
python notebooks/eval.py
python notebooks/vit_eval.py
python core/inference/parameter_tune/resnet_tune.py
python core/inference/parameter_tune/vit_tune.py

Enrollment and Usage

• Use enroll.py to add new users embeddings to the system.

from recognition.core.inference.enrollment_service import enroll_user

"""
name : str : The name of the user to enroll.
role : str : The role of the user (e.g., "student", "staff").
folder : str : The folder path where the user's images are stored for enrollment.
model_name : str : The name of the model to use for enrollment (e.g., "inception_resnet_v1", "vit").
weight_path : str : The path to the model weights file to use for enrollment.
"""

enroll_user(name, role, folder, model_name, weight_path, database)

Database Schema/Abstraction

from abc import ABC, abstractmethod


class Identities(ABC):

    @abstractmethod
    def get_by_name(self, name):
        raise NotImplementedError

    @abstractmethod
    def get_all(self):
        raise NotImplementedError

    @abstractmethod
    def update(self, identity):
        raise NotImplementedError

    @abstractmethod
    def create(self, identity):
        raise NotImplementedError

    @abstractmethod
    def close(self):
        raise NotImplementedError

Using Vision from another project (submodule usage)

If this repo is added as a submodule, import and start Vision like this:

import asyncio
from IPython.display import display
from PIL import Image
from recognition.interaction.vision import Vision


async def handle_vision_event(event_data: dict):
    """Callback triggered whenever user state changes."""
    if not event_data:
        return

    event_type = event_data.get("event")
    name = event_data.get("name", "unknown")
    role = event_data.get("role")

    if event_type == "user_changed":
        print(f"[VISION EVENT] User Changed -> Name: {name} | Role: {role}")

        if name == "unknown":
            await system.say(
                "Hello there! I don't believe we've met yet. Could you tell me your name?"
            )
            response = await system.listen(timeout=6.0)
            if response:
                await system.say(f"Nice to meet you, {response}!")

        elif role == "student":
            await system.set_robot_role_mode("student")
            await system.say(
                f"Welcome back, {name}! "
                f"I am ready as your personal development tutor today. "
                f"What would you like to work on?"
            )

        elif role == "staff":
            await system.set_robot_role_mode("staff")
            await system.say(
                f"Hello {name}. "
                f"Staff assistance mode activated. "
                f"How can I assist you today?"
            )

    elif event_type == "user_left":
        print(f"[VISION EVENT] User Left -> Name: {name}")
        if name and name != "unknown":
            await system.say(f"Goodbye {name}, have a nice day!")


system = Vision(host="127.0.0.1", callback=handle_vision_event, use_webcam=True)

# In Jupyter/async environments, create a task instead of blocking.
monitoring_task = asyncio.create_task(system.start_monitoring())

await asyncio.sleep(3)
if system.processed_image:
    display(Image.open(system.processed_image))

Vision constructor

• Vision(host="localhost", callback=None, use_webcam=False)

Event payloads

• user_changed: includes name, role, score, previous_user
• user_left: includes name, role

Add this repo as a Git submodule

From your parent project root:

git submodule add <YOUR_REPO_URL> external/dissertation-codebase
git commit -m "Add dissertation codebase submodule"

Clone later with submodules:

git clone --recurse-submodules <PARENT_PROJECT_URL>

If already cloned without submodules:

git submodule update --init --recursive

Use it in Python (example):

import sys
from pathlib import Path

sys.path.append(str(Path("external/dissertation-codebase").resolve()))
from recognition.interaction.vision import Vision

Notes

• SLURM shell scripts (train_*.sh, eval_*.sh, tune_*.sh) are cluster-oriented and may need local path/env updates.
• Ensure your dataset paths and model checkpoints in recognition/config.py match your machine.