ai-regression-agent 0.2.1

Backend — Setup and Run Guide

Follow these steps from the repository root. Commands shown for both Windows (cmd/Powershell) and Unix-like shells where applicable.

1) Clone the repository

git clone https://CTS-AI@dev.azure.com/CTS-AI/IDC_Family_Safety_Agent/_git/IDC_Family_Safety_Agent
cd IDC_Family_Safety_Agent

2) Enter the backend directory

cd backend

3) Prerequisites

• Install Python 3.12 and ensure python/python3 is on your PATH.
• (Recommended) Create and activate a virtual environment:
  • Windows (cmd): python -m venv .venv & \.venv\Scripts\activate
  • Unix/macOS: python3 -m venv .venv & source .venv/bin/activate

4) Install Python dependencies

pip install -r requirements.txt

If you need to install some packages manually, you can also run:

pip install playwright python-dotenv openai azure-storage-blob flask flask-cors pandas jinja2 openpyxl msal requests tldextract transformers

5) Install Playwright browsers

Run this from the same environment where Playwright is installed:

playwright install

On some systems you may also need playwright install-deps.

6) Configure environment variables

• Copy the example environment file and update values:
  • Windows (cmd): copy .env_example .env
  • Unix-like: cp .env_example .env
• Edit .env and set the required values (examples):
  • ADO_PLAN_ID
  • ADO_SUITE_ID
  • APP_PATH (local path to the Microsoft Teams app executable)
  • ADO_ORG
  • ADO_PROJECT
  • ADO_PAT
  • Optional: LOGIN_EMAIL, LOGIN_PASSWORD, Azure OpenAI/Blob settings, etc.

7) Run the agent

From the backend directory run:

python run_natural_tests.py

This will start the agent and execute tests according to the configured ADO plan/suite or local test JSON fallback.

Notes

• Ensure APP_PATH points to the correct Teams executable for your OS.
• Updating ADO settings (ADO_PLAN_ID, ADO_SUITE_ID, ADO_ORG, ADO_PROJECT, ADO_PAT) changes which test cases are fetched.
• Generated reports, screenshots and videos are placed under backend/reporting/reports/consolidated reports. If Azure Blob is configured, report links may be uploaded and shared.

Report formatting flags

• If HCLTech_TestCaseId=false, the final report shows only the Test Case ID column.
• If HCLTech_TestCaseId=true, the final report shows both Test Case ID and HCLTech ADO ID columns.