Check Websites HTTP Status Codes with httpcheck

• Name: httpcheck (CLI) / onSite (Menu Bar App)
• Current Version: 1.5.1 (security patch for 1.5.0: Async I/O & Configuration)
• Next Version: 1.6.0 (Monitoring Mode, target Q1 2027)
• Programming Language: Python 3.10+
• Author: Thomas Juul Dyhr
• Purpose: Advanced HTTP status checker with monitoring capabilities
• Development Status: Active - See ROADMAP.md for development plan
• PyPI: https://pypi.org/project/httpcheck/

🚀 Release Status

httpcheck v1.5.1 has been released! 🎉

✅ v1.5.1 (security patch): installing httpcheck now also requires patched versions of urllib3, certifi, idna, anyio and h11, so an install into an existing environment cannot keep vulnerable copies. No behaviour changes.

✅ NEW in v1.5.0:

• Configuration Files: User-defined defaults in ~/.httpcheck.toml and ./.httpcheck.toml (the project file overrides the user file); CLI flags always take precedence
• Async I/O (experimental): --async uses an httpx-based concurrent path. In the benchmark against v1.4.3 it is not faster than -f at the same --workers; for speed, raise -f --workers
• Python 3.10+: v1.5.0 drops Python 3.9 support (pip on Python 3.9 keeps installing 1.4.3); Python 3.13 and 3.14 are tested in CI
• Retry fix: with --follow-redirects http-only or https-only, retries now work (1.4.3 reported the site as failed without retrying)

✅ v1.4.2-1.4.3 (Published to PyPI):

• Structured Logging: --debug, --log-file, --log-json flags for production monitoring
• Comprehensive Testing: 297 tests with 88% coverage (CLI: 94%)
• Professional Documentation: 24-page documentation with ReadTheDocs theme
• Performance Benchmarks: 18 automated regression tests
• PyPI Published: Available via pip install httpcheck
• Automated Publishing: GitHub Actions with Trusted Publishers (OIDC)

✅ v1.4.0-1.4.1 Foundation:

• Modular Architecture: 8 specialized modules
• Enhanced Security: Enterprise-grade input validation with pip-audit clean
• Multiple Output Formats: JSON and CSV support
• Advanced Request Control: Custom headers, SSL options

🔭 Next: v1.6.0 (target Q1 2027)

• Monitoring Mode: Continuous site monitoring with notifications
• Enhanced UX: Colored output and improved progress reporting

See detailed plans in:

• DEVELOPMENT_PLAN.md - Executive overview and timeline
• DEVELOPMENT.md - Technical implementation guide
• TODO.md - Current prioritized task list
• ROADMAP.md - Long-term vision through v2.0.0

📦 Installation

From PyPI (Recommended)

pip install httpcheck

From Source

git clone https://github.com/docdyhr/httpcheck.git
cd httpcheck
pip install -e .

With Optional Dependencies

# For macOS menu bar app
pip install httpcheck[macos]

# For development
pip install httpcheck[dev]

Upgrade to Latest Version

pip install --upgrade httpcheck

usage:

httpcheck [-h] [-t] [--disable-tld-checks] [--tld-warning-only]
          [--update-tld-list] [--tld-cache-days TLD_CACHE_DAYS]
          [-q | -v | -c | -f] [--timeout TIMEOUT] [--retries RETRIES]
          [--workers WORKERS] [--file-summary]
          [--comment-style {hash,slash,both}]
          [--follow-redirects {always,never,http-only,https-only}]
          [--max-redirects MAX_REDIRECTS] [--show-redirect-timing]
          [--output {table,json,csv}] [--version] [site ...]

positional arguments

site return http status codes for one or more websites

optional arguments

-h, --help show this help message and exit -t, --tld check if domain is in global list of TLDs --disable-tld-checks disable TLD validation checks --tld-warning-only show warnings for invalid TLDs without failing --update-tld-list force update of the TLD list from publicsuffix.org --tld-cache-days TLD_CACHE_DAYS number of days to keep the TLD cache valid (default: 30) -q, --quiet only print errors -v, --verbose increase output verbosity -c, --code only print status code -f, --fast fast check wtih threading --timeout TIMEOUT set the timeout for each request --retries RETRIES set the number of retries for each request --retry-delay DELAY delay in seconds between retry attempts (default: 1.0) --workers WORKERS set the number of worker threads -H, --header HEADER add custom HTTP header (can be used multiple times) --no-verify-ssl disable SSL certificate verification --file-summary show summary of file parsing results (valid URLs, comments, etc.) --comment-style {hash,slash,both} comment style to recognize: hash (#), slash (//), or both (default: both) --follow-redirects {always,never,http-only,https-only} control redirect following behavior (default: always) --max-redirects MAX_REDIRECTS maximum number of redirects to follow (default: 30) --show-redirect-timing show detailed timing for each redirect in the chain --output {table,json,csv} output format: table (default), json, or csv --version show program's version number and exit

additional information

enter sites in url or 'no' url form: 'httpcheck duckduckgo.com' read sites from a file: 'httpcheck @domains.txt'.

output formats:

• Table (default): Human-readable columnar output
• JSON: Machine-parsable format for automation
• CSV: Comma-separated values for spreadsheets

Example: 'httpcheck google.com github.com --output json'

List of HTTP status codes

installation

clone repo and install requirements.

clone the repository

git clone https://github.com/docdyhr/httpcheck
cd httpcheck/

install requirements

python3 -m pip install . --user

On macOS, install terminal-notifier for notifications:

brew install terminal-notifier

alternative install

set up a virtual environment with venv.

python3 -m venv .venv
source .venv/bin/activate
pip install -e .

or

mv httpcheck.py ~/bin/httpcheck # any other bin folder
chmod + ~/bin/httpcheck

features

• Check HTTP status codes for one or more websites
• Support for macOS notifications on failures
• NEW: Menu bar app for continuous monitoring
  • Live status indicator in menu bar (🟢/🔴)
  • Background checking with configurable intervals
  • Rich native macOS notifications with actions
  • Click notifications to open failing sites
  • Easy site management through menu interface
• Advanced redirect tracking and control
  • Follow all redirects, none, or limit by protocol (HTTP/HTTPS)
  • Configure maximum number of redirects to follow
  • View detailed per-hop timing information in redirect chains
• Enhanced TLD validation
  • Caches TLD list for better performance
  • Auto-updates from Public Suffix List
  • Configurable cache duration
  • Optional warning-only mode
• Enhanced file input handling
  • Support for comments using # or // style
  • Handles inline comments
  • Automatically strips whitespace
  • Detailed file parsing statistics
• Threading support for faster checks
• Progress bar for multiple site checks
• Configurable timeouts and retries
• Pipe and file input support
• NEW: Multiple output formats
  • JSON format for easy parsing and automation
  • CSV format for spreadsheet integration
  • Table format (default) for human readability
  • Verbose mode includes redirect details in all formats
• NEW: Request customization options
  • Custom HTTP headers support with -H flag
  • Configurable retry delay between attempts
  • SSL certificate verification control
  • Enhanced timeout configuration

usage examples

python3 httpcheck.py https://duckduckgo.com

returns ex. 'duckduckgo.com 200'

python from venv

python httpcheck.py -v https://api.github.com/invalid

returns verbose output ex. '[-] api.github.com --> Client errors: 404 Not Found'

installed as binary

httpcheck -q @domains.txt

only returns http status code errors ex. 'api.github.com 404'

httpcheck -c 1.1.1.1

status code only ex. '200'

notifications

On macOS, the tool will show notifications when sites fail checks. For sites with failures:

• Shows failure count
• Lists failed sites (when less than 10)
• Groups notifications to avoid duplicates

advanced usage examples

Control redirect behavior:

# Don't follow any redirects
httpcheck --follow-redirects never example.com

# Only follow redirects to HTTP URLs (not HTTPS)
httpcheck --follow-redirects http-only example.com

# Only follow redirects to HTTPS URLs (not HTTP)
httpcheck --follow-redirects https-only example.com

# Show detailed timing for each redirect
httpcheck --show-redirect-timing http://example.com

# Limit redirect chain length
httpcheck --max-redirects 5 example.com

# Show timing for each redirect hop
httpcheck -v --show-redirect-timing example.com

tld validation examples

Control TLD validation behavior:

# Enable TLD validation
httpcheck -t example.com

# Force update of the TLD list from Public Suffix List
httpcheck -t --update-tld-list example.com

# Warn about invalid TLDs but don't fail
httpcheck -t --tld-warning-only example.com

# Disable TLD checks completely
httpcheck --disable-tld-checks example.com

# Set cache expiration to 60 days
httpcheck -t --tld-cache-days 60 example.com

output format examples

Export results in different formats:

# JSON output for single site
httpcheck google.com --output json

# JSON output for multiple sites with verbose details
httpcheck google.com github.com --output json -v

# CSV output for spreadsheet import
httpcheck @domains.txt --output csv

# CSV with full details including redirects
httpcheck @domains.txt --output csv -v

# Process JSON output with jq
httpcheck @sites.txt --output json | jq '.[] | select(.status != "200")'

# Save CSV report with date
httpcheck @sites.txt --output csv -v > report_$(date +%Y%m%d).csv

# Filter errors only in JSON format
httpcheck @sites.txt --output json -q | jq '.[] | .domain'

request customization examples

Add custom headers and control SSL verification:

# Add authorization header
httpcheck -H "Authorization: Bearer token123" api.example.com

# Multiple custom headers
httpcheck -H "User-Agent: MyBot 1.0" -H "Accept: application/json" api.example.com

# Disable SSL verification for self-signed certificates
httpcheck --no-verify-ssl https://self-signed.example.com

# Custom timeout and retry configuration
httpcheck --timeout 30 --retries 5 --retry-delay 2.0 slow.example.com

# Combine with other options
httpcheck -H "API-Key: secret" --output json --no-verify-ssl https://api.example.com

configuration file

httpcheck reads defaults from ~/.httpcheck.toml (user-wide) and/or ./.httpcheck.toml (project-local). Project-level settings override user-level settings; any CLI flag always takes final precedence.

# ~/.httpcheck.toml

[defaults]
timeout = 10.0
retries = 3
follow_redirects = "always"   # always | never | http-only | https-only
output_format = "table"       # table | json | csv
verify_ssl = true
workers = 20
retry_delay = 0.5
max_redirects = 10

[headers]
User-Agent = "mybot/1.0"
Accept = "application/json"

[notifications]
enabled = true
on_failure = true
sound = "Ping"

Unknown keys are silently ignored with a UserWarning. Invalid TOML falls back to no config (also a UserWarning).

file input examples

Create a domains file with comments and use it with httpcheck:

# domains.txt - Example domains file
google.com  # Search engine
facebook.com // Social media
twitter.com
// Commented out: example.com

# Basic file usage
httpcheck @domains.txt

# Show file parsing statistics
httpcheck --file-summary @domains.txt

# Only recognize # style comments
httpcheck --comment-style hash @domains.txt

# Only recognize // style comments
httpcheck --comment-style slash @domains.txt

onSite - menu bar app (macOS)

For continuous monitoring with a native macOS experience, use onSite - the menu bar app:

installation

# Install additional dependencies for menu bar app
pip install rumps pyobjc-framework-Cocoa py2app

# Run the menu bar app directly
python3 httpcheck_menubar.py

building a standalone app

Create a native macOS app bundle:

# Build the app (creates dist/onSite.app)
python setup.py py2app

# Copy to Applications folder
cp -r "dist/onSite.app" /Applications/

# Run from Applications or Spotlight
open "/Applications/onSite.app"

menu bar app features

• Live Status Indicator: Menu bar shows ⚡ (white lightning - all good/checking), 🔴⚡ (red + lightning - failures)
• Badge Count: Shows number of failed sites next to the icon
• Rich Notifications: Native macOS notifications with:
  • Site down alerts with status codes
  • Recovery notifications when sites come back online
  • Click actions to open failing sites in browser
  • Custom sounds for different alert types
• Background Monitoring: Configurable check intervals (default 15 minutes, minimum 60 seconds)
• Site Management: Add, remove, and edit monitored sites through menu
• Settings: Configure check intervals, clear failed sites, view logs
• Configuration Storage: Sites and settings saved to ~/.httpcheck/

menu bar app usage

1. Add Sites: Click menu bar icon → "Add Site..." → Enter URL (automatically checks status)
2. Manual Check: Click "Check Now" for immediate status check
3. Auto-Check: Toggle automatic background checking on/off (default: 15 minutes)
4. View Status: Click on menu to see all sites with status indicators
5. Settings: Adjust check interval, clear failed sites, view logs

keyboard shortcuts and tips

• Menu bar shows white lightning ⚡ for normal status, red lightning 🔴⚡ for failures
• Failed site count appears as badge number
• Notifications include action buttons (Check Now, View, Dismiss)
• Clicking notifications opens the failing site in your default browser
• Logs are stored in ~/Library/Logs/onSite/onsite.log following macOS best practices
• Log rotation: 5MB max size, 5 backup files maintained automatically
• Logs viewable through Console.app or "View logs" menu item
• Configuration files stored in ~/.httpcheck/ for easy backup

history

checking one or more websites by constantly opening a browser seems a waste of time. Providing a cli tool that can be used in a bash shell, pipe or for scheduling tasks in cron, notifying the user, seems the way to go. The python requests library seems perfect for the job.

todo

TODO.md

license

MIT