Skip to main content

WebGrep Tweet

Grep Web pages and their resources.

PyPi Python Versions Read The Docs Build Status Security Coverage Status License

This self-contained tool relies on the well-known grep tool for grepping Web pages. It binds nearly every option of the original tool and also provides additional features like deobfuscating Javascript or appyling OCR on images before grepping downloaded resources.

$ pip install webgrep-tool

Quick Start

  1. Help
$ webgrep --help
usage: webgrep [OPTION]... PATTERN [URL]...

Search for PATTERN in each input URL and its related resources (images, scripts and style sheets).
By default,
- resources are NOT downloaded
- response HTTP headers are NOT included in grepping ; use '--include-headers'
- PATTERN is a basic regular expression (BRE) ; use '-E' for extended (ERE)
Important note: webgrep does not handle recursion (in other words, it does not spider additional web pages).
Examples:
 webgrep example http://www.example.com     # will only grep on HTML code
 webgrep -r example http://www.example.com  # will only grep on LOCAL images, ...
 webgrep -R example http://www.example.com  # will only grep on ALL images, ...

Regexp selection and interpretation:
 -e REGEXP, --regexp REGEXP
                       use PATTERN for matching
 -f FILE, --file FILE  obtain PATTERN from FILE
 -E, --extended-regexp
                       PATTERN is an extended regular expression (ERE)
 -F, --fixed-strings   PATTERN is a set of newline-separated fixed strings
 -G, --basic-regexp    PATTERN is a basic regular expression (BRE)
 -P, --perl-regexp     PATTERN is a Perl regular expression
 -i, --ignore-case     ignore case distinctions
 -w, --word-regexp     force PATTERN to match only whole words
 -x, --line-regexp     force PATTERN to match only whole lines
 -z, --null-data       a data line ends in 0 byte, not newline

Miscellaneous:
 -s, --no-messages     suppress error messages
 -v, --invert-match    select non-matching lines
 -V, --version         print version information and exit
 --help                display this help and exit
 --verbose             verbose mode
 --keep-files          keep temporary files in the temporary directory
 --temp-dir TMP        define the temporary directory (default: /tmp/webgrep)

Output control:
 -m NUM, --max-count NUM
                       stop after NUM matches
 -b, --byte-offset     print the byte offset with output lines
 -n, --line-number     print line number with output lines
 --line-buffered       flush output on every line
 -H, --with-filename   print the file name for each match
 -h, --no-filename     suppress the file name prefix on output
 --label LABEL         use LABEL as the standard input filename prefix
 -o, --only-matching   show only the part of a line matching PATTERN
 -q, --quiet, --silent
                       suppress all normal output
 --binary-files TYPE   assume that binary files are TYPE;
                       TYPE is 'binary', 'text', or 'without-match'
 -a, --text            equivalent to --binary-files=text
 -I                    equivalent to --binary-files=without-match
 -L, --files-without-match
                       print only names of FILEs containing no match
 -l, --files-with-match
                       print only names of FILEs containing matches
 -c, --count           print only a count of matching lines per FILE
 -T, --initial-tab     make tabs line up (if needed)
 -Z, --null            print 0 byte after FILE name

Context control:
 -B NUM, --before-context NUM
                       print NUM lines of leading context
 -A NUM, --after-context NUM
                       print NUM lines of trailing context
 -C NUM, --context NUM
                       print NUM lines of output context

Web options:
 -r, --local-resources
                       also grep local resources (same-origin)
 -R, --all-resources   also grep all resources (even non-same-origin)
 --include-headers     also grep HTTP headers
 --cookie COOKIE       use a session cookie in the HTTP headers
 --referer REFERER     provide the referer in the HTTP headers

Proxy settings (by default, system proxy settings are used):
 -d, --disable-proxy   manually disable proxy
 --http-proxy HTTP     manually set the HTTP proxy
 --https-proxy HTTPS   manually set the HTTPS proxy

Please report bugs on GitHub: https://github.com/dhondta/webgrep/issues/new
  1. Example
$ webgrep -R Welcome https://github.com
      Welcome home, <br>developers

Resource Handlers

Definitions:

  • Resource (what is being processed): Web page, images, Javascript, CSS
  • Handler (how a resource is processed): CSS unminifying, OCR, deobfuscation, EXIF data retrieval, ...

The handlers are defined in the HANDLERS SECTION of src/webgrep/__main__.py (see Extending WebGrep). Currently available handlers :

  1. Images
  • EXIF: using exiftool
  • Steganography: using steghide (with a blank password)
  • Strings: using strings
  • OCR: using tesseract
  1. Scripts
  • Javascript beautifying and deobfuscation: using jsbeautifier
  1. Styles
  • Unminifying: using regular expressions

Note: images found in the CSS files are also processed.

Back to top

Metadata

Release files for webgrep-tool 1.20

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for webgrep-tool 1.20
File Size Uploaded Provenance Actions
webgrep_tool-1.20.tar.gz 203.4 kB With provenance Details for webgrep_tool-1.20.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for webgrep-tool 1.20
File Interpreter ABI Platform Actions
webgrep_tool-1.20-py3-none-any.whl Python 3 none any Details for webgrep_tool-1.20-py3-none-any.whl

Total release size: 244.0 kB

Release files / webgrep_tool-1.20.tar.gz

Download URL webgrep_tool-1.20.tar.gz
Size 203.4 kB
Tags Source
SHA-256 checksum
How to use checksums
a07609cee727580e254b4837b71225a200d56dfe69cdc4b8d48ffe19be5acf89
BLAKE2b-256 checksum
How to use checksums
f1c48d157374d40928d74a4db4fb360ebcad376e270d653c71c38046b747e581
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

Release files / webgrep_tool-1.20-py3-none-any.whl

Download URL webgrep_tool-1.20-py3-none-any.whl
Size 40.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ad9ef94b38a1d2aee3243dc7993c844ac417632a7bff0bf9561e1afb6c6d8254
BLAKE2b-256 checksum
How to use checksums
422edc4805ed4a7a5380f089373559945e4e7440d8f23f3bceea1d890961c919
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

Security for webgrep-tool 1.20

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for webgrep_tool-1.20.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page