Skip to main content

A tool for downloading files with filenames matching user-specified regex patterns which are linked to on a user-provided URL

Project description

DoRePy - (Do)wnload (Re)gex (Py)thon

DoRePy (pronounced like doe-ray-pee) is your go-to script for automating the download of files from a webpage that match a specific regex pattern. Fed up with manually sifting through pages to download files? DoRePy has got your back!

Features

  • Regex Pattern Matching: Use the power of regular expressions to target exactly the files you need.
  • Retry Logic: Network hiccup? No problem. DoRePy retries failed downloads, respecting rate limits like a well-mannered netizen.

Getting Started

Prerequisites

  • Python 3
  • Requests: pip install requests
  • BeautifulSoup: pip install beautifulsoup4

Installation

Clone this repository or simply download dorepy.py to your local machine:

git clone https://github.com/CillySu/DoRePy/dorepy.git

Usage

Navigate to the directory containing dorepy.py and run:

python dorepy.py [URL] [PATTERN]

Where:

[URL] is the webpage URL from which you want to download files. [PATTERN] is the regex pattern that matches the file names you want to download.

Example:

python dorepy.py "http://example.com" "\.pdf$"

This command downloads all PDF files which are linked to on http://example.com.

Contributing

Feel like DoRePy missed a beat? Fork the repo, add your spin, and submit a pull request. All contributions are welcome!

License

Distributed under the MIT License. See LICENSE for more information.

A Note on Responsible Use

Please use DoRePy wisely and respect website terms of service and your local laws as applicable.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

DoRePy-0.1.1.tar.gz (3.4 kB view details)

Uploaded Source

Built Distribution

DoRePy-0.1.1-py3-none-any.whl (3.9 kB view details)

Uploaded Python 3

File details

Details for the file DoRePy-0.1.1.tar.gz.

File metadata

  • Download URL: DoRePy-0.1.1.tar.gz
  • Upload date:
  • Size: 3.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.5

File hashes

Hashes for DoRePy-0.1.1.tar.gz
Algorithm Hash digest
SHA256 0c432a87d164b5dc47dd961b2ff61f23187d4ff555ab4fb7145eab685bc5fb54
MD5 9a02edec4d89c2352d284379b45b6def
BLAKE2b-256 29243806e1965bde16c7c90e425a370fb45c16eef5855910d327157c5d2b1114

See more details on using hashes here.

File details

Details for the file DoRePy-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: DoRePy-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 3.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.5

File hashes

Hashes for DoRePy-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 4e65a0408cad4e2a5e3ea3706e20944fac5261b4cf7f6117c124a620111b2437
MD5 6a3a446be0456c43e5a4c6d9cbadb294
BLAKE2b-256 872d9db861a10977bd86a1c93b06b76b0953869ddc979b95336e896afa085438

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page