webchk 1.2.0

A command-line tool for checking HTTP status codes and response headers of URLs

webchk

webchk is a command-line tool developed in Python 3 for checking the HTTP status codes and response headers of URLs. It accepts one or more URLs as arguments. Furthermore, a sitemap URL can be passed using the -p option to download its content, extract the URLs and check their statuses.

Installation

webchk is available on PyPI and can be installed using pip with the following command:

$ pip install webchk

Webchk does not require any 3rd party packages to run. So it can also be cloned from GitHub and run as a module:

$ git clone https://github.com/amgedr/webchk.git
$ cd webchk
$ python3 -m webchk

Usage

webchk [-h] [-i INPUT] [-o OUTPUT] [-p] [-a] [-l] [-s] [-f] [-v]
             [urls [urls ...]]

positional arguments:
  urls

optional arguments:
  -h, --help                   show this help message and exit
  -i INPUT, --input INPUT      Read input from a file
  -o OUTPUT, --output OUTPUT   Save output to a file
  -p, --parse                  Follow links listed in .xml URLs
  -l, --list                   Print URLs without checking them
  -v, --version                Print the version number

Examples

Check a list of URLs from a file (one URL per line):

$ webchk -i urls.txt

Check the status of a sitemap file and all the URLs listed in it:

$ webchk -p http://example.com/sitemap.xml

List the URLs in a file without checking their HTTP status:

$ webchk -li urls.txt

Check the URLs in a file and .xml files in it:

$ webchk -pi urls.txt

List the URLs in a file and .xml files in it:

$ webchk -pli urls.txt

List the URLs in a sitemap without checking their status:

$ webchk -lp http://example.com/sitemap.xml

History

1.2.0 (2022-02-19)

• Default HTTP method is now HEAD

• Added –get to switch to HTTP GET

• Added –agent to set the User-Agent

• Added –auth for adding Authorization header

• Prevent timeouts less than 1

• Display results immediately instead of at the end

1.1.0 (2022-02-05)

• Handle connection related errors

• Improve reporting of HTTP redirects

• Added –timeout or -t to modify respose deadline

1.0.4 (2020-08-29)

• Fix crash when an URL check times out

• Fix –output command-line option

1.0.3 (2020-07-10)

• Modify the Python versions tox tests

• Fix failing unit tests

1.0.2 (2020-05-08)

• Add Python versions 3.7 and 3.8 to the list of tested versions

1.0.1 (2020-05-08)

• Fixed: Parsing sitemaps enters an endless loop

• Fixed: Parsing a URL that does not exists exits with an unhandled exception

1.0.0 (2018-12-06)

• Linked to http://codehill.com/projects/webchk/ instead of readthedocs.io

0.3.0 (2018-03-24)

• Run each check in its own thread

0.2.1 (2017-12-19)

• Fixed: Status code description not being displayed

• Improved PyPI and GitHub README

0.2.0 (2017-12-14)

• Code refactoring

• Created setup.py