Full-featured web UI for monitoring and controlling Scrapyd servers cluster, with Scrapy log analysis and visualization supported
Project description
ScrapydWeb: Full-featured web UI for monitoring and controlling Scrapyd servers cluster, with Scrapy log analysis and visualization supported
Features
-
Multinode Scrapyd Servers
- Group, filter and select any numbers of nodes
- Execute command on multinodes with one click
-
Scrapy Log Analysis
- Stats collection
- Progress visualization
- Logs categorization
-
All Scrapyd API Supported
- Deploy project, Run Spider, Stop job
- List projects/versions/spiders/running_jobs
- Delete version/project
-
Others
- Basic auth for web UI
Maintainer
Installation
To install ScrapydWeb, simply use pip:
$ pip install scrapydweb
Start Up
Run "scrapydweb -h" to get help, and a config file named "scrapydweb_settings.py" would be copied to the working directory, then you can custom config with it.
$ scrapydweb
Visit http://127.0.0.1:5000
Screenshot
-
Overview
-
Dashboard
-
Log Analysis
-
Stats collection
-
Progress visualization
-
Logs categorization
-
-
Deploy Project
-
Run Spider
-
Manage Projects
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapydweb-0.9.5.tar.gz
(575.8 kB
view hashes)
Built Distribution
scrapydweb-0.9.5-py3-none-any.whl
(602.2 kB
view hashes)
Close
Hashes for scrapydweb-0.9.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dde52b9b6cd2119dc977b8c60fb533bd901657523316a1b93f3e2b0796c671b7 |
|
MD5 | 44ed477fb8580a32b24870bde637d2d9 |
|
BLAKE2b-256 | 2fcfba3f64036cde7c0202cae827f4b8e53b245b7760c6eee63761cc951e096e |