This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description
1 - INTRODUCTION
----------------
The program "arsespyder" is a Web Crawler that enables crawling a URL and inspect the links contained in that web recursively, so that for a provided URL, the crawler downloads the links of type <a href="http://whatever">whatever</a> contained on it.
For each of them the operations is repeated, until the level of search provided by parameter is achieved. If not provided, the crawling depth levels is, by default, 3.

2 - APPLICATION USAGE
---------------------
Usage of the application is as follows:

$ ./arsespyder.py --help
usage: arsespyder.py [-h] [-v] [-n NUMBER_OF_LEVELS] url

Internet Crawler

positional arguments:
url URL to crawl

optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-n NUMBER_OF_LEVELS, --number-of-levels NUMBER_OF_LEVELS
Crawling depth

NOTE 1 - The only mandatory parameter is the URL to crawl.
NOTE 2 - If number of levels is not provided, by default, the NUMBER_OF_LEVELS is 3.

3 - OUTPUT FORMAT
-----------------
arsespyder Web crawler will dump those links up to the depth crawling level specified, in the next format:

$ ./arsespyder.py -n3 http://arsespyder.dyndns.org/index.html
* http://arsespyder.dyndns.org/test/l1_p1.html
* http://arsespyder.dyndns.org/test/l1_p2.html
* http://arsespyder.dyndns.org/test/l1_p3.html
** http://arsespyder.dyndns.org/test/l2_p1_p1.html
** http://arsespyder.dyndns.org/test/l2_p1_p2.html
** http://arsespyder.dyndns.org/test/l2_p2_p1.html
** http://arsespyder.dyndns.org/test/l2_p2_p2.html
*** http://arsespyder.dyndns.org/l3_p1_p1_p1.html
*** http://arsespyder.dyndns.org/l3_p1_p1_p2.html
*** http://arsespyder.dyndns.org/l3_p1_p1_p3.html
*** http://arsespyder.dyndns.org/l3_p1_p2_p1.html
*** http://arsespyder.dyndns.org/l3_p1_p2_p2.html
*** http://arsespyder.dyndns.org/l3_p1_p2_p3.html

Where:
* http://... are level 1 links (existing in HTML code URL specified parameter)
** http://... are level 2 links (existing in HTML code of level 1 links)
*** http://... are level 3 links (existing in HTML code of level 2 links)

4 - CODE DOCUMENTATION
----------------------
Code documentation is contained under "doc" folder. Main .html file is pyarsespyder.html:

$ tree doc/
doc/
├── pyarsespyder.geturl.html
├── pyarsespyder.html
└── pyarsespyder.validateurl.html

5 - INSTALATION
---------------
Check INSTALL file

Release History

Release History

0.0.3

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
arsespyder-0.0.3.linux-i686.tar.gz (4.5 kB) Copy SHA256 Checksum SHA256 any Dumb Binary Sep 22, 2013
arsespyder-0.0.3.tar.gz (3.9 kB) Copy SHA256 Checksum SHA256 Source Sep 22, 2013

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting