Skip to main content

Full-featured web UI for monitoring and controlling your Scrapyd servers

Project description

ScrapydWeb: Full-featured web UI for monitoring and controlling Scrapyd servers

Feature Support

  • Multinode Scrapyd Servers

    • Group, filter and select any numbers of nodes
    • Execute command on multinodes with one click
  • Scrapy Log Analysis

    • Collect statistics
    • Show crawling progress with chart
    • Extract key logs
  • All Scrapyd API supported

    • Deploy project, Run Spider, Stop job
    • List projects/versions/spiders/running_jobs
    • Delete version/project

Maintainer

Installation

To install ScrapydWeb, simply use pip:

$ pip install scrapydweb

Start Up

Run "scrapydweb -h" to get help, and a config file named "scrapydweb_settings.py" would be copied to the working directory, then you can custom config with it

$ scrapydweb

Visit http://127.0.0.1:5000

Screenshot

  • Overview overview
  • Dashboard dashboard
  • Manage Projects manage
  • Run Spider run

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapydweb-0.9.1.tar.gz (573.3 kB view hashes)

Uploaded source

Built Distribution

scrapydweb-0.9.1-py3-none-any.whl (596.8 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page