Skip to main content

Eek, a [web] spider.

Project description

Eek is a web crawler that outputs metadata about a website in CSV format.

Installation

$ pip install eek

Usage

usage: eek [-h] [–graph] [–delay SECONDS] [–grep PATTERN] [-i] URL

eek recursively crawls a website, outputing metadata about each page in CSV format.

positional arguments:
  URL                The base URL to start the crawl

optional arguments:
  -h, --help         show this help message and exit
  --graph            output a graphviz digraph of links instead of CSV
                     metadata
  --delay SECONDS    Time, in seconds, to wait in between fetches. Defaults to
                     0.
  --grep PATTERN     Print urls containing PATTERN (a python regular
                     expression).
  -i, --ignore-case  Ignore case. Only valid with --grep

Example:

eek http://example.com/

To save output to a file, use redirection

eek http://example.com/ > ~/some_file.csv

To slow down crawling, use --delay=[seconds]

Project details


Release history Release notifications

This version
History Node

1.0.2

History Node

1.0.1

History Node

1.0.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
eek-1.0.2.tar.gz (7.4 kB) Copy SHA256 hash SHA256 Source None Jul 11, 2014

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page