Web spider and fetcher
Project description
Installation
$ pip install spiderfetch
$ spiderfetchUsage
Fetching
Fetch all urls matching 2008.*.ogg from a page:
$ spiderfetch http://www.fosdem.org/2008/media/video 2008.*oggTo dump the urls to a file instead of fetching:
$ spiderfetch http://www.fosdem.org/2008/media/video 2008.*ogg --dump > urlsSpidering
Spider a site to depth 3 while pausing 2 seconds between fetches. The urls that will be considered when spidering must match .*:
$ spiderfetch --host http://en.wikipedia.org --depth 3 --pause 2 '.*'
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spiderfetch-0.4.9.tar.gz
(18.0 kB
view details)
Built Distribution
File details
Details for the file spiderfetch-0.4.9.tar.gz.
File metadata
- Download URL: spiderfetch-0.4.9.tar.gz
- Upload date:
- Size: 18.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 6598628fcd573d24279530abcf5ad1a142b4eba462f869fbcd8360b0640f0560 |
|
| MD5 | b26cfe275009ca5974555677be48fc90 |
|
| BLAKE2b-256 | 6052a6b6b7dd217c5290d47cfc2279f2a80fbd6f7f9d2b60afa1268b2edbd8ff |
File details
Details for the file spiderfetch-0.4.9-py2.py3-none-any.whl.
File metadata
- Download URL: spiderfetch-0.4.9-py2.py3-none-any.whl
- Upload date:
- Size: 24.0 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | a8d26d909299edfe00064ecc4263e86834637b0ce447a0202bdbadd79cd065c5 |
|
| MD5 | 61c8a8f4195108b69421e647d6558dd1 |
|
| BLAKE2b-256 | 8f910194d07bd954e067019153d07dc9a7a0160331ea793aac6a2168765fe32f |
