A tipical crawler, returns a list of urls
Project description
# Copyright (C) 2011 Diego Pardilla Mata
#
# This file is part of SpiderBOY.
#
# SpiderBOY is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
This is the SpiderBoy crawler.
It has (among others) the following major features
Search URL's from another URL and with a depth indicated
Installation:
See INSTALL file for information on how to install SpiderBOY
Starting SpiderBOY:
run from the command line: `spiderboy.py [-h] [-n NUMBER_OF_LEVELS] [url]'
run `spiderboy.py -h' for more startup options
More information:
http://code.sidelab.es/projects/diegopardilla/wiki
Written by:
see AUTHORS for more information
Reporting Bugs:
Please report bugs to our mailing list or our bugtracker:
munikes@members.fsf.org
pardilla@members.fsf.org
http://code.sidelab.es/projects/diegopardilla/issues/new
Contacting us:
http://packages.python.org/SpiderBOY (homepage)
munikes@members.fsf.org (mUniKeS)
pardilla@members.fsf.org (Diego Pardilla)
#
# This file is part of SpiderBOY.
#
# SpiderBOY is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
This is the SpiderBoy crawler.
It has (among others) the following major features
Search URL's from another URL and with a depth indicated
Installation:
See INSTALL file for information on how to install SpiderBOY
Starting SpiderBOY:
run from the command line: `spiderboy.py [-h] [-n NUMBER_OF_LEVELS] [url]'
run `spiderboy.py -h' for more startup options
More information:
http://code.sidelab.es/projects/diegopardilla/wiki
Written by:
see AUTHORS for more information
Reporting Bugs:
Please report bugs to our mailing list or our bugtracker:
munikes@members.fsf.org
pardilla@members.fsf.org
http://code.sidelab.es/projects/diegopardilla/issues/new
Contacting us:
http://packages.python.org/SpiderBOY (homepage)
munikes@members.fsf.org (mUniKeS)
pardilla@members.fsf.org (Diego Pardilla)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
SpiderBOY-1.0.0.tar.gz
(4.6 kB
view hashes)
Built Distributions
SpiderBOY-1.0.0.linux-x86_64.exe
(70.3 kB
view hashes)
SpiderBOY-1.0.0-py2.7.egg
(8.5 kB
view hashes)
Close
Hashes for SpiderBOY-1.0.0.linux-x86_64.exe
Algorithm | Hash digest | |
---|---|---|
SHA256 | ff68834eb4c9cf7a077e2104ccb2c2a9cb6bdc750c073e951f6b32513b6ba66b |
|
MD5 | dc0d353160f569530b4af76d58e0903e |
|
BLAKE2b-256 | 7a279f398ec5f9309033bd6431a4d4c4fc0b68a284faac461946269ba644dab0 |