Spider trap for testing crawlers
Project description
An http server that responds with an ever-expanding web of links. Use it to test, benchmark, and confuse web crawlers.
Implemented as a simple python script with no dependencies other than the standard library. Works with python 2.7 and python 3.
Installation
$ pip install spidertrap
Usage
spidertrap [-h] [-p PORT] [--fanout FANOUT] optional arguments: -h, --help show this help message and exit -p PORT, --port PORT [9999] --fanout FANOUT Number of links to create on each page [2]
Licence
MIT
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spidertrap-0.1.tar.gz
(1.5 kB
view hashes)