spidertrap

Spider trap for testing crawlers

Project description

An http server that responds with an ever-expanding web of links. Use it to test, benchmark, and confuse web crawlers.

Implemented as a simple python script with no dependencies other than the standard library. Works with python 2.7 and python 3.

Installation

$ pip install spidertrap

Usage

spidertrap [-h] [-p PORT] [--fanout FANOUT]

optional arguments:
  -h, --help            show this help message and exit
  -p PORT, --port PORT  [9999]
  --fanout FANOUT       Number of links to create on each page [2]

Licence

MIT

Project details

Release history Release notifications | RSS feed

This version

0.1.1

Apr 6, 2014

0.1

Apr 5, 2014

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spidertrap-0.1.1.tar.gz (1.5 kB view details)

Uploaded Apr 6, 2014 Source

File details

Details for the file spidertrap-0.1.1.tar.gz.

File metadata

Download URL: spidertrap-0.1.1.tar.gz
Upload date: Apr 6, 2014
Size: 1.5 kB
Tags: Source
Uploaded using Trusted Publishing? No

File hashes

Hashes for spidertrap-0.1.1.tar.gz
Algorithm	Hash digest
SHA256	`a86e75692f09c4760b1d6934e059b3ae17dbc111f78762422680d393b2639472`
MD5	`fc6f26892fe79dfceea370fbcf994788`
BLAKE2b-256	`2660071eb36d543477a37fe8a7df70a25a32a67c151360f054b9b4acb918325c`