Web spider and fetcher
Project description
Installation
$ pip install spiderfetch
$ spiderfetchUsage
Fetching
Fetch all urls matching 2008.*.ogg from a page:
$ spiderfetch http://www.fosdem.org/2008/media/video 2008.*oggTo dump the urls to a file instead of fetching:
$ spiderfetch http://www.fosdem.org/2008/media/video 2008.*ogg --dump > urlsSpidering
Spider a site to depth 3 while pausing 2 seconds between fetches. The urls that will be considered when spidering must match .*:
$ spiderfetch --host http://en.wikipedia.org --depth 3 --pause 2 '.*'
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spiderfetch-0.4.9.tar.gz
(18.0 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file spiderfetch-0.4.9.tar.gz.
File metadata
- Download URL: spiderfetch-0.4.9.tar.gz
- Upload date:
- Size: 18.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6598628fcd573d24279530abcf5ad1a142b4eba462f869fbcd8360b0640f0560
|
|
| MD5 |
b26cfe275009ca5974555677be48fc90
|
|
| BLAKE2b-256 |
6052a6b6b7dd217c5290d47cfc2279f2a80fbd6f7f9d2b60afa1268b2edbd8ff
|
File details
Details for the file spiderfetch-0.4.9-py2.py3-none-any.whl.
File metadata
- Download URL: spiderfetch-0.4.9-py2.py3-none-any.whl
- Upload date:
- Size: 24.0 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a8d26d909299edfe00064ecc4263e86834637b0ce447a0202bdbadd79cd065c5
|
|
| MD5 |
61c8a8f4195108b69421e647d6558dd1
|
|
| BLAKE2b-256 |
8f910194d07bd954e067019153d07dc9a7a0160331ea793aac6a2168765fe32f
|
