Skip to main content

Quick web spider and traffic generator

Project description

Spinneret

Spinneret is a web spider and dynamic load generator for testing, benchmarking, and warming caches.

Setup

git clone git@github.com:steder/spinneret.git
cd spinneret
mkdir -p ~/.virtualenvs
virtualenv ~/.virtualenvs/spinneret
pip install -r requirements

Generating a sitemap

$ ./bin/spider

Because this can take a while I’ve included a sample sitemap.yaml that can be used to immediately generate traffic.

Using that sitemap to generate requests against a site

./bin/traffic http://godzilla.threadless.com

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spinneret-0.0.1.tar.gz (4.3 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page