Skip to main content

A web spider for collecting specific data across a set of configured sites

Project description


Parker is a Python-based web spider for collecting specific data
across a set of configured sites.

Non-Python requirements:

- Redis - for task queuing and visit tracking
- libxml - for HTML parsing of pages


Install using ``pip``::

$ pip install parker


To configure Parker, you will need to install the configuration
files in a suitable location for the user running Parker. To do
this, use the ``parker-config`` script. For example::

$ parker-config ~/.parker

This will install the configuration in your homedir and will output
the related environment variable for you to set in your ``.bashrc``.



- Added handling for a PARKER_CONFIG environment variable, allowing
users to specify where configuration files are loaded from.

- Added the ``parker-config`` script to install default configuration
files to a passed location. Also prints out an example PARKER_CONFIG
environment variable to add to your profile files.

- Updated documentation to use proper reStructuredText files.

- Add a CHANGES file to track updates.


- Bug fix to see if RST in ASCII fixes issues on PyPI.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
Parker-0.4.1.tar.gz (137.7 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page