Skip to main content

Full-featured web UI for monitoring and controlling your Scrapyd servers

Project description

ScrapydWeb: Full-featured web UI for monitoring and controlling Scrapyd servers

Feature Support

  • Multinode Scrapyd Servers

    • Group, filter and select any numbers of nodes
    • Execute command on multinodes with one click
  • Scrapy Log Analysis

    • Collect statistics
    • Show crawling progress with chart
    • Extract key logs
  • All Scrapyd API supported

    • Deploy project, Run Spider, Stop job
    • List projects/versions/spiders/running_jobs
    • Delete version/project



To install ScrapydWeb, simply use pip:

$ pip install scrapydweb

Start Up

Run "scrapydweb -h" to get help, and a config file named "" would be copied to the working directory, then you can custom config with it

$ scrapydweb



  • Overview overview

  • Dashboard dashboard

  • Log Analysis

    • Statistics log_stats

    • Crawling progress log_chart

    • Key Logs log_extracted

  • Deploy Project deploy

  • Run Spider run

  • Manage Projects manage

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapydweb-0.9.2.tar.gz (573.6 kB view hashes)

Uploaded source

Built Distribution

scrapydweb-0.9.2-py3-none-any.whl (596.9 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page