Hybrid Browser AI Agent for LinkedIn Easy Apply

A Python-based browser automation AI agent built with Playwright, a Finite State Machine (FSM), Page Object Model (POM), and an LLM-driven Semantic Matching & Form Interpretation Engine with a strict Human-in-the-Loop (HITL) approval gate.

Architecture Overview

d:\Gravity\
├── config/
│   ├── candidate_profile.json  # Single source of truth for candidate data
│   └── settings.py             # Pydantic BaseSettings (.env loader)
├── models/
│   ├── profile.py              # Pydantic models for Candidate Profile
│   ├── job.py                  # JobCardSummary and JobDetails schemas
│   ├── evaluation.py           # MatchBreakdown and JobEvaluationResult schemas
│   ├── form.py                 # FormField, FormStep, ApplicationSummary schemas
│   └── application.py          # Database ApplicationRecord and Status
├── services/
│   ├── profile_loader.py       # Profile reader & dynamic field calculation
│   ├── db_service.py           # SQLite/PostgreSQL persistence & deduplication
│   ├── llm_service.py          # Multi-backend LLM client & NLP heuristic engine
│   └── resume_service.py       # Resume path manager & reportlab PDF generator
├── automation/
│   └── pages/
│       ├── base_page.py        # Playwright POM base with anti-detection & typing jitter
│       ├── login_page.py       # Auth state verification & manual checkpoint polling
│       ├── job_search_page.py  # Easy Apply search navigation & job card extraction
│       ├── job_details_page.py # Description extraction & Easy Apply launcher
│       └── easy_apply_modal.py # Modal step traversal, dynamic filling, review
├── agents/
│   ├── job_evaluator.py        # LLM semantic fit scorer & experience analysis
│   ├── form_agent.py           # Dynamic form question interpreter & HITL routing
│   └── fsm_runner.py           # Finite State Machine orchestrator
├── tests/                      # Pytest automated test suite
├── main.py                     # Rich CLI runner & dashboard
├── requirements.txt            # Python dependencies
└── .env.example                # Example environment variables

Finite State Machine (FSM) Lifecycle

stateDiagram-v2
    [*] --> INIT
    INIT --> LAUNCH_BROWSER: Load profile & init database
    LAUNCH_BROWSER --> CHECK_LOGIN: Launch persistent Chromium context
    CHECK_LOGIN --> CHECK_LOGIN: Pause & wait if CAPTCHA / 2FA / Login needed
    CHECK_LOGIN --> SEARCH_JOBS: Authenticated
    SEARCH_JOBS --> EXTRACT_CARDS: Search with f_AL=true (Easy Apply)
    EXTRACT_CARDS --> EVALUATE_FIT: Filter out applied jobs
    EVALUATE_FIT --> NEXT_JOB: Score < min_score OR Experience Reject
    EVALUATE_FIT --> FILL_FORM: Score >= min_score -> Launch Modal
    FILL_FORM --> AWAIT_APPROVAL: Reached final review step
    AWAIT_APPROVAL --> SUBMIT: User Approved [y]
    AWAIT_APPROVAL --> NEXT_JOB: User Skipped [n]
    SUBMIT --> TRACK: Click submit
    TRACK --> NEXT_JOB: Persist to applications.db
    NEXT_JOB --> EXTRACT_CARDS: Next unapplied card
    NEXT_JOB --> [*]: Max jobs reached or finished

Key Features & Adherence to Requirements

1. Finite State Machine & Page Object Model:

   • Clean separation between Playwright UI interactions (automation/pages/), agent decision logic (agents/), data storage (services/), and schema contracts (models/).

2. Stealth & Persistent Browser Context:

   • Uses Chromium in non-headless mode by default (HEADLESS=false).
   • Uses persistent profile directory (.browser_context/) to preserve cookies, sessions, and tokens.
   • Zero Credential Storage: LinkedIn passwords are never stored in code, .env, or logs.
   • No Automated Solvers: If LinkedIn prompts with 2FA, OTP, or CAPTCHA, the agent halts, prints an alert to the terminal, and waits for manual completion in the open browser before resuming.

3. Candidate Profile as Single Source of Truth:

   • Located at config/candidate_profile.json.
   • All downstream calculations (compensation, experience years, notice period, skill lookups) dynamically read from this profile without hardcoded values.

4. Semantic LLM Fit Scoring:

   • Evaluates:
     • Experience Match: Candidate has 4 years; if job asks for 5–8 years, score is downgraded proportionally; if >8 years or Director/Lead, hard rejection.
     • Skill Match: Semantic clustering recognizing synonyms (e.g. RestAssured/Postman <-> API testing, Selenium/Playwright <-> Web automation).
     • Role Match: Checks relevance against preferred roles (QA Automation Engineer, SDET, etc.).
   • Skips postings when overall_score < minimum_match_score (70%).

5. Dynamic Form Filling:

   • Maps standard personal details, current/expected CTC, 60 days notice period, and skill-specific experience years.
   • Handles dropdowns, radio groups, and file uploads.
   • Flags sensitive/ambiguous declarations (clearance, disability, citizenship) for explicit HITL prompt.

6. Human-in-the-Loop (HITL) Gate:

   • Pauses on the final review step of the modal.
   • Renders a Rich summary table (Job Title, Company, Match Score, File Uploaded).
   • Requires explicit user approval ([y] Approve & Submit / [n] Skip / [q] Quit) before clicking the submit button.

7. Application Tracking & Deduplication:

   • Async SQLite database (applications.db) with an applications table.
   • Queries by job_id or job_url + company before opening any job card to avoid duplicates.

Installation & Setup

1. Install Dependencies:

   pip install -r requirements.txt
   

2. Initialize Playwright Browser:

   playwright install chromium
   

3. Configure Environment (Optional):

   cp .env.example .env
   

   Add your OPENAI_API_KEY or GEMINI_API_KEY if you want live cloud LLM reasoning. If left empty, the built-in deterministic NLP extraction and semantic scoring engine will run offline automatically.

Usage Guide

1. Dry Run (Recommended for testing without submitting)

Simulates the workflow, extracts cards, computes fit scores, fills forms up to the final review step, and tests the HITL prompt without submitting:

python main.py --keyword "QA Automation Engineer" --location "Hyderabad" --max-jobs 3 --dry-run

2. Live Apply Mode

Executes the live workflow with non-headless browser:

python main.py --keyword "SDET" --location "Hyderabad" --min-score 70 --max-jobs 5

3. CLI Command Options

--keyword      Job title or search keyword (default: "QA Automation Engineer")
--location     Search location (default: "Hyderabad")
--max-jobs     Maximum applications to process (default: 10)
--min-score    Minimum fit score required to proceed (default: 70.0)
--headless     Run Chromium in headless mode (default: False)
--dry-run      Fill forms to review step without clicking final submit
--auto-approve Bypass interactive HITL prompt at review step
--profile      Custom path to candidate_profile.json
--log-level    Logging level [DEBUG, INFO, WARNING, ERROR] (default: INFO)

Running the Automated Test Suite

Execute pytest across all test modules:

pytest tests/ -v