Skip to main content

Crawl a website and audit SEO, accessibility, performance & broken links — zero dependencies.

Project description

██████╗  ██████╗ ██╗     ██╗   ██╗
██╔══██╗██╔════╝ ██║     ██║   ██║
██████╔╝██║  ███╗██║     ██║   ██║
██╔═══╝ ██║   ██║██║     ██║   ██║
██║     ╚██████╔╝███████╗╚██████╔╝
╚═╝      ╚═════╝ ╚══════╝ ╚═════╝
PGLU · site-doctor

🩺 site-doctor

Crawl any website and audit SEO · Accessibility · Performance · Broken links — from your terminal.

An agency-grade website health checker in pure Python. Get a scored report in seconds, output JSON for dashboards, or gate your CI/CD on a minimum health score.

Python Zero deps CI License


  site-doctor — https://example.com
  Pages audited: 12  ·  Links checked: 140

  Health Score: 82.4/100
  Grade: B — good

  SEO            ████████████████░░  88.0/100
  Accessibility  ██████████████░░░░  79.0/100
  Performance    ██████████████████  96.0/100
  Links          ████████████████░░  88.0/100

  Top issues
  [error] a11y: 4/9 images missing an alt attribute. (7 pages)
  [warn]  seo:  Meta description is 182 chars (>160). (3 pages)
  ...
  Broken links (2)
  ✗ 404  https://example.com/old-page

📖 New here? Read the full User Manual (USAGE.md) — install, first audit, every option, and troubleshooting (including the "command not recognized" fix).

✨ What it checks

Category Examples
🔍 SEO title, meta description, single H1, canonical, Open Graph + og:image, Twitter Card, <meta charset>, favicon, viewport, JSON-LD, thin content
Accessibility <html lang>, alt text, vague/empty link text, form labels, heading order — each tagged with its WCAG success criterion
Performance measured load time, HTML size, script/stylesheet count, large inline JS, request count, gzip/brotli & Cache-Control headers
🔒 Security HTTPS, security headers (HSTS, CSP, X-Content-Type-Options, X-Frame-Options, Referrer-Policy), mixed content
🔗 Broken links parallel HTTP checks of internal and external links, with status codes
🗂️ Site-wide duplicate <title> and duplicate meta descriptions across pages

Plus a weighted overall health score and letter grade.

Smart link checking: genuinely broken links (4xx/5xx) hurt your score, but links that merely can't be verified — bot-blocked (LinkedIn 999, social WAFs) or unreachable — are reported separately and don't unfairly tank your score. Crawling is concurrent, and ANSI colors work even in legacy Windows consoles.

🚀 Install

pip install site-doctor

Or run from source (no install): PYTHONPATH=src python -m sitedoctor <url>

🕹️ Usage

site-doctor example.com                  # full audit (crawls up to 20 pages)
site-doctor https://mysite.com --max-pages 50 --max-depth 3
site-doctor mysite.com --json > report.json    # machine-readable
site-doctor mysite.com --no-external           # skip external link checks (faster)
site-doctor mysite.com --fail-under 80         # exit 1 if score < 80  (great for CI)

site-doctor mysite.com --sitemap               # audit every URL in /sitemap.xml
site-doctor mysite.com --html report.html      # client-ready HTML report
site-doctor mysite.com --md report.md          # Markdown report
site-doctor mysite.com --csv issues.csv        # CSV of all issues & links (spreadsheets)

Options

Flag Description
--max-pages N Max pages to crawl (default 20)
--max-depth N Max crawl depth (default 2)
--max-links N Max links to check (default 200)
--timeout N Per-request timeout in seconds (default 15)
--sitemap Audit URLs from sitemap.xml instead of crawling
--sitemap-url URL Use an explicit sitemap URL (implies --sitemap)
--html PATH Also write a self-contained HTML report
--md PATH Also write a Markdown report
--csv PATH Also write a CSV of all issues & links
--junit PATH Also write a JUnit XML report (for CI test dashboards)
--ignore CODE Suppress a check by code (repeatable), e.g. --ignore og-missing
--min-seo / --min-a11y / --min-performance / --min-security / --min-links N Per-category CI gates
--no-robots Ignore robots.txt (it's respected by default)
--no-external Don't check external links
--browser-ua Send a desktop-browser User-Agent (for sites that block bots)
--user-agent UA Custom User-Agent string
--insecure Skip TLS certificate verification (fixes cert errors)
--fail-under N Exit non-zero if health score < N
--json / --no-color JSON output / plain text
--no-banner Hide the PGLU banner

🤖 Gate your deploys on site health (GitHub Actions)

- name: Audit site health
  run: |
    pip install site-doctor
    site-doctor https://your-site.com --fail-under 80 --min-security 70 --junit site.xml

The build fails if the overall score — or any category you gate on — regresses below your threshold. Emit --junit XML for your CI's test dashboard.

🧱 How it works

Pure standard library: a breadth-first crawler (urllib), an html.parser-based extractor, pure-function check rules, weighted scoring, and a ThreadPoolExecutor-powered parallel link checker. No browser, no third-party packages.

crawler → parser → checks (seo/a11y/perf) → scoring → report → render
                                  links checker ↗

📦 Publishing (maintainers)

Publishing to PyPI is a manual action via publish.yml using Trusted Publishing (no API tokens). One-time setup on pypi.org: Your account → Publishing → Add a pending publisher with project site-doctor, owner aashishbharti04, repo site-doctor, workflow publish.yml, environment pypi. Then run the workflow from the Actions tab.

🛠️ Develop

pip install -e ".[dev]"
pytest -q

🤝 Contributing

New checks, output formats, or fixes are welcome — see CONTRIBUTING.md.

📄 License

MIT © Aashish Bharti

⭐ Star it if site-doctor saved you a manual audit.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

site_doctor-0.4.1.tar.gz (30.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

site_doctor-0.4.1-py3-none-any.whl (28.4 kB view details)

Uploaded Python 3

File details

Details for the file site_doctor-0.4.1.tar.gz.

File metadata

  • Download URL: site_doctor-0.4.1.tar.gz
  • Upload date:
  • Size: 30.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for site_doctor-0.4.1.tar.gz
Algorithm Hash digest
SHA256 58ae3fe180140fd8cb35999f65cb89734070738e60fe771a6e48bf49842d441a
MD5 2df1dee5279ddb01f344d8bff0699b67
BLAKE2b-256 1fe181cac952f08fb07a8dd811eabdad7005893c450aa4a23a25cf0db80c6bb4

See more details on using hashes here.

Provenance

The following attestation bundles were made for site_doctor-0.4.1.tar.gz:

Publisher: publish.yml on aashishbharti04/site-doctor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file site_doctor-0.4.1-py3-none-any.whl.

File metadata

  • Download URL: site_doctor-0.4.1-py3-none-any.whl
  • Upload date:
  • Size: 28.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for site_doctor-0.4.1-py3-none-any.whl
Algorithm Hash digest
SHA256 f2b4d124c70fdbea7ce30343790503ba67d2d455e717b63618d519ffa4124f8d
MD5 a46cfb02c9117d96ad1e04e35e875108
BLAKE2b-256 bbb07381923b037d00f0bda74e041e23513b4ef2aace8f406f58e1f494afbfb1

See more details on using hashes here.

Provenance

The following attestation bundles were made for site_doctor-0.4.1-py3-none-any.whl:

Publisher: publish.yml on aashishbharti04/site-doctor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page