This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Eek, a [web] spider.

Project Description

Eek is a web crawler that outputs metadata about a website in CSV format.

Installation

$ pip install eek

Usage

usage: eek [-h] [–graph] [–delay SECONDS] [–grep PATTERN] [-i] URL

eek recursively crawls a website, outputing metadata about each page in CSV format.

positional arguments:
  URL                The base URL to start the crawl

optional arguments:
  -h, --help         show this help message and exit
  --graph            output a graphviz digraph of links instead of CSV
                     metadata
  --delay SECONDS    Time, in seconds, to wait in between fetches. Defaults to
                     0.
  --grep PATTERN     Print urls containing PATTERN (a python regular
                     expression).
  -i, --ignore-case  Ignore case. Only valid with --grep

Example:

eek http://example.com/

To save output to a file, use redirection

eek http://example.com/ > ~/some_file.csv

To slow down crawling, use --delay=[seconds]

Release History

Release History

This version
History Node

1.0.2

History Node

1.0.1

History Node

1.0.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
eek-1.0.2.tar.gz (7.4 kB) Copy SHA256 Checksum SHA256 Source Jul 11, 2014

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting