# spidey.py
> Web spiders are usually disliked by websites, but useful for recursive API/page downloads for offline analysis.
## Installation
> Pypi Location: https://pypi.python.org/pypi/spidey.py
- Using Pypi - `pip install spidey`
## Usage
> Run `spidey` for Detailed help.
- `spidey --dir NEW_DIR --filter DOMAIN --url URL [--base BASE_URL]`
- `spidey --dir NEW_DIR --filter DOMAIN --url URL --max MAX_DOWNLOADS`
- Example - `spidey --dir test --filter 'www.google.com' --url 'https://www.google.com/' --max 20`
### More Examples
```
spidey \
-d test \
-f 'www.google.com' \
-u 'https://www.google.com/' \
-b 'https://www.google.com/' \
-hh '{"Accept" : "application/json"}' \
-n 2 \
-m 10 \
-s 5
```
```
spidey \
--dir test \
--filter 'www.google.com' \
--url 'https://www.google.com/'' \ \
--base 'https://www.google.com/
--headers '{"Accept" : "application/json"}' \
--depth 2 \
--max 10 \
--sleep 5
```
> Web spiders are usually disliked by websites, but useful for recursive API/page downloads for offline analysis.
## Installation
> Pypi Location: https://pypi.python.org/pypi/spidey.py
- Using Pypi - `pip install spidey`
## Usage
> Run `spidey` for Detailed help.
- `spidey --dir NEW_DIR --filter DOMAIN --url URL [--base BASE_URL]`
- `spidey --dir NEW_DIR --filter DOMAIN --url URL --max MAX_DOWNLOADS`
- Example - `spidey --dir test --filter 'www.google.com' --url 'https://www.google.com/' --max 20`
### More Examples
```
spidey \
-d test \
-f 'www.google.com' \
-u 'https://www.google.com/' \
-b 'https://www.google.com/' \
-hh '{"Accept" : "application/json"}' \
-n 2 \
-m 10 \
-s 5
```
```
spidey \
--dir test \
--filter 'www.google.com' \
--url 'https://www.google.com/'' \ \
--base 'https://www.google.com/
--headers '{"Accept" : "application/json"}' \
--depth 2 \
--max 10 \
--sleep 5
```
1 maintainer
Release files for spidey.py 0.4.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| spidey.py-0.4.5.tar.gz | 4.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| spidey.py-0.4.5-py2-none-any.whl | Python 2 | none | any | Details |
Total release size: 10.1 kB
Release files / spidey.py-0.4.5.tar.gz
| Download URL | spidey.py-0.4.5.tar.gz |
|---|---|
| Size | 4.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ad995d1d6f4ddafdfe26f314d5d48926c410b6ef2c1663150a6a32905eac7c12
|
|
BLAKE2b-256 checksum How to use checksums |
f3aad79122826bb541ae57d27284dc1a4120eb752db04c22cad816b2c87265f0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / spidey.py-0.4.5-py2-none-any.whl
| Download URL | spidey.py-0.4.5-py2-none-any.whl |
|---|---|
| Size | 6.2 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
3e958927065cf9a26dfd3dfcf0007103db01ead2b3d6657ce93db2b5bd25cc02
|
|
BLAKE2b-256 checksum How to use checksums |
63aba640c7c0e233ec3a7908e97a76d12dd7bb55e5026547744095e2e55a13bd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
