This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!
Project Description

Arachnado

Arachnado is a tool to crawl a specific website. It provides a Tornado-based HTTP API and a web UI for a Scrapy-based crawler.

License is MIT.

Install

Arachnado requires Python 2.7. To install Arachnado use pip:

pip install arachnado

To install Arachnado with MongoDB support use this command:

pip install arachnado[mongo]

Run

To start Arachnado execute arachnado command:

arachnado

and then visit http://0.0.0.0:8888 (or whatever URL is configured).

To see available command-line options use

arachnado –help

Arachnado can be configured using a config file. Put it to one of the common locations (‘/etc/arachnado.conf’, ‘~/.config/arachnado.conf’ or ‘~/.arachnado.conf’) or pass the file name as an argument when starting the server:

arachnado --config ./my-config.conf

For available options check https://github.com/TeamHG-Memex/arachnado/blob/master/arachnado/settings/defaults.conf.

Development

To build Arachnado static assets node.js + npm are required. Install all JavaScript requirements using npm - run the following command from the repo root:

npm install

then rebuild static files (we use Webpack):

npm run build

or auto-build static files on each change during development:

npm run watch

Changes

0.2 (2015-08-07)

Initial release.

Release History

Release History

0.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
arachnado-0.2-py2-none-any.whl (156.3 kB) Copy SHA256 Checksum SHA256 2.7 Wheel Aug 7, 2015
arachnado-0.2.tar.gz (140.0 kB) Copy SHA256 Checksum SHA256 Source Aug 7, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting