Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

WebScraper

  • This program can scrap data from websites using different scrapers, and send an email when matches/ changes deadening on the scraper used
  • There are 2 types of scrapers:
    • Generic: Can scrap any website, but might not be as exact
    • Specific: Can scrap only specific websites, but will be more exact

Generic Scrapers

  • Text
  • Diff

Specific Scrapers

  • Cars.com

How to use

Text

  1. Set these specific env variables
  2.  SCRAPER=text # Scraper to use
     URL=<URL> # URL to scrape
     TEXT=<TEXT> # Text to look for
    
  3. Ensure all other required env variables are set

Diff

  1. Set these specific env variables
  2.  SCRAPER=diff # Scraper to use
     URL=<URL> # URL to scrape
     PERCENTAGE=<PERCENTAGE_DIFF> # Percentage difference to look for
    
  3. Ensure all other required env variables are set

Cars.com

  1. Set these specific env variables
  2.  SCRAPER=cars_com # Scraper to use
     URL=https://www.cars.com/shopping/results/ # URL to scrape, must be on the results page, for a specific search
    
  3. Ensure all other required env variables are set

Required env variables

SLEEP_TIME_SEC= # Time to sleep between each scrape
SENDER_EMAIL= # Email to send from
FROM_EMAIL= # Name to send from i.e. '"Web Scraper" <no-reply@jstockley.com>'
RECEIVER_EMAIL= # Email to send to
PASSWORD= # Password for the sender's email
SMTP_SERVER= # SMTP server to use
SMTP_PORT= # SMTP port to use
TLS= # True/False to use TLS

Running multiple of the same scraper

To run 2+ scrapers of the same type, i.e. 2 diff scrapers, make sure the host folder mapping is different Ex:

  diff-scraper-1:
    image: jnstockley/web-scraper:latest
    volumes:
      - ./diff-scraper-1-data/:/app/data/
    environment:
      - TZ=America/Chicago
      - SCRAPER=diff
      - URL=https://google.com
      - PERCENTAGE=5
      - SLEEP_TIME_SEC=21600

  diff-scraper-2:
    image: jnstockley/web-scraper:latest
    volumes:
      - ./diff-scraper-2-data/:/app/data/
    environment:
      - TZ=America/Chicago
      - SCRAPER=diff
      - URL=https://yahoo.com
      - PERCENTAGE=5
      - SLEEP_TIME_SEC=21600

Metadata

Release files for js-web-scraper 20261028233629.dev0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for js-web-scraper 20261028233629.dev0
File Size Uploaded Provenance Actions
js_web_scraper-20261028233629.dev0.tar.gz 19.3 kB With provenance Details for js_web_scraper-20261028233629.dev0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for js-web-scraper 20261028233629.dev0
File Interpreter ABI Platform Actions
js_web_scraper-20261028233629.dev0-py3-none-any.whl Python 3 none any Details for js_web_scraper-20261028233629.dev0-py3-none-any.whl

Total release size: 40.6 kB

Release files / js_web_scraper-20261028233629.dev0.tar.gz

Download URL js_web_scraper-20261028233629.dev0.tar.gz
Size 19.3 kB
Tags Source
SHA-256 checksum
How to use checksums
9e76e99b02e60a66aa317e4c3de520ad87fa23a9067c99bfc3c36339f5adfc49
BLAKE2b-256 checksum
How to use checksums
4ec538de3f47fca88e04997780550ed7ff0d4ccf6959e39cdcc2eb8ea5c075a3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.

Transparency log

Release files / js_web_scraper-20261028233629.dev0-py3-none-any.whl

Download URL js_web_scraper-20261028233629.dev0-py3-none-any.whl
Size 21.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0d035bc33b04790d5fc5490af896b5ac28d9c81c08ffc2713093b93cde32efc4
BLAKE2b-256 checksum
How to use checksums
41390f6a270cf3fe58cd87fdd81c67edc0eed4f3c06fafc1a9ee277bea7ee7a1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.

Transparency log

Security for js-web-scraper 20261028233629.dev0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for js_web_scraper-20261028233629.dev0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page