Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

A tipical crawler, returns a list of urls

Project Description
# Copyright (C) 2011 Diego Pardilla Mata
#
# This file is part of SpiderBOY.
#
# SpiderBOY is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

This is the SpiderBoy crawler.

It has (among others) the following major features

Search URL's from another URL and with a depth indicated

Installation:

See INSTALL file for information on how to install SpiderBOY

Starting SpiderBOY:

run from the command line: `spiderboy.py [-h] [-n NUMBER_OF_LEVELS] [url]'
run `spiderboy.py -h' for more startup options

More information:

http://code.sidelab.es/projects/diegopardilla/wiki

Written by:

see AUTHORS for more information

Reporting Bugs:

Please report bugs to our mailing list or our bugtracker:
munikes@members.fsf.org
pardilla@members.fsf.org
http://code.sidelab.es/projects/diegopardilla/issues/new

Contacting us:

http://packages.python.org/SpiderBOY (homepage)
munikes@members.fsf.org (mUniKeS)
pardilla@members.fsf.org (Diego Pardilla)
Release History

Release History

This version
History Node

1.0.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
SpiderBOY-1.0.0.linux-x86_64.exe (70.3 kB) Copy SHA256 Checksum SHA256 any Windows Installer Feb 21, 2015
SpiderBOY-1.0.0-py2.7.egg (8.5 kB) Copy SHA256 Checksum SHA256 2.7 Egg Feb 21, 2015
SpiderBOY-1.0.0.tar.gz (4.6 kB) Copy SHA256 Checksum SHA256 Source Feb 21, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting