av-wpa (Web Page Analyzer)

A CLI utility for downloading web pages, downloading page resources, extracting structured data such as ld+json microdata and inline JavaScript variables for analyzis.

Features

• Downloading web page by url
• Extracting data: ld-json structured microdata, and JavaScript state objects, list of resources.
• Resource Downloading: Save page HTML and download associated web assets directly to a directory.

Installation

Option 1: Install via PyPI

Using standard pip:

pip install av-wpa

Using astral-uv:

uv add av-wpa

Option 2: From Source

git clone https://github.com/Avazart/av-wpa.git
cd wpa
uv sync

How to Run

Command Line Interface (CLI)

Run via installed entry point:

python -m av-wpa [OPTIONS] URL

Or run directly via python module:

python -m av_wpa [OPTIONS] URL

Or execute using uv without manual venv activation:

uv run -m av_wpa [OPTIONS] URL

Command Line Arguments & Options

Positional Arguments

• url — Target web page URL to process (requared).

Options & Flags

• -d, --directory PATH — Specify the download directory path for saving output files.
• -a, --analyze — Enable analysis mode for parsing and extracting page structured data.
• -r, --resources — Enable resource downloading to fetch page assets.
• -h, --help — Show help message and exit.

Execution Examples

1. Basic Analysis

Analyze a target web page:

python -m av-wpa https://example.com -a

2. Download Page Resources to a Specific Directory

Download web page resources into a designated folder:

python -m av-wpa https://example.com -r -d ./data

3. Analyze and Download Resources Together

python -m av-wpa https://example.com -a -r -d ./data