Arachnado
Arachnado is a tool to crawl a specific website. It provides a Tornado-based HTTP API and a web UI for a Scrapy-based crawler.
License is MIT.
Install
Arachnado requires Python 2.7. To install Arachnado use pip:
pip install arachnado
To install Arachnado with MongoDB support use this command:
pip install arachnado[mongo]
Run
To start Arachnado execute arachnado command:
arachnado
and then visit http://0.0.0.0:8888 (or whatever URL is configured).
To see available command-line options use
arachnado –help
Arachnado can be configured using a config file. Put it to one of the common locations (‘/etc/arachnado.conf’, ‘~/.config/arachnado.conf’ or ‘~/.arachnado.conf’) or pass the file name as an argument when starting the server:
arachnado --config ./my-config.conf
For available options check https://github.com/TeamHG-Memex/arachnado/blob/master/arachnado/settings/defaults.conf.
Development
Source code: https://github.com/TeamHG-Memex/arachnado
Issue tracker: https://github.com/TeamHG-Memex/arachnado/issues
To build Arachnado static assets node.js + npm are required. Install all JavaScript requirements using npm - run the following command from the repo root:
npm install
then rebuild static files (we use Webpack):
npm run build
or auto-build static files on each change during development:
npm run watch
Changes
0.2 (2015-08-07)
Initial release.
Release files for arachnado 0.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| arachnado-0.2.tar.gz | 140.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| arachnado-0.2-py2-none-any.whl | Python 2 | none | any | Details |
Total release size: 296.3 kB
Release files / arachnado-0.2.tar.gz
| Download URL | arachnado-0.2.tar.gz |
|---|---|
| Size | 140.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9125eba9e911caba1434462606eed212c4e4998720dc04a67669bffc4961a9fd
|
|
BLAKE2b-256 checksum How to use checksums |
47644b5f813d98099aff09c763450b56cf1b6e8fcbf8b7178ada7f6bea6e4fda
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / arachnado-0.2-py2-none-any.whl
| Download URL | arachnado-0.2-py2-none-any.whl |
|---|---|
| Size | 156.3 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
62271c8c3b240e60d90d57228faada0dcd1254cc1eab749060a097409d19272a
|
|
BLAKE2b-256 checksum How to use checksums |
5e5c0301f345ef205f1680dd595c2eb700af3b7a61b2626ee679d524a343b4c7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
