A service for running Scrapy spiders, with an HTTP API
Project description
==========
ScrapydArt
==========
.. image:: https://secure.travis-ci.org/scrapy/scrapyd.svg?branch=master
:target: http://travis-ci.org/scrapy/scrapyd
.. image:: https://codecov.io/gh/scrapy/scrapyd/branch/master/graph/badge.svg
:target: https://codecov.io/gh/scrapy/scrapyd
==========
项目演示地址
==========
http://scrapydart.porters.vip
==========
拒绝裸奔
==========
ScrapydArt在Scrapyd基础上新增了权限验证、筛选过滤、排序、数据统计以及排行榜等功能,并且有了更强大的API。
scrapyd是世界最优秀的爬虫框架scrapy官方提供的部署控管理平台,它提供了爬虫任务调用的api、爬虫运行日志功能。
但它并不提供权限验证功能,意味着只要你部署到服务器,任何人都可以访问它并且控制你的爬虫。它也不提供数据统计的功能,你不可能知道你部署的爬虫有多少、也不知道它们运行了多少次、谁的运行时间最长、更别说排行了。官方开发人员关注的重点是api功能性而非界面,所以界面(甚至可以说没有界面)不谈美观。
==========
ScrapydArt安装
==========
pip install scrapydart
==========
ScrapydArt启动
==========
scrapydart
==========
ScrapydArt-web界面
==========
http://localhost:6800
--------------------------------------------------
更多使用方法和ScrapydArt的详细介绍请阅读ScrapydArt文档
--------------------------------------------------
https://scrapydart.readthedocs.io/zh/latest/
--------------------
下图为ScrapydArt界面
--------------------
.. image:: https://github.com/dequinns/ScrapydArt/blob/master/images/scrapydart-home.png
.. image:: https://github.com/dequinns/ScrapydArt/blob/master/images/scrapydart-jobs.png
ScrapydArt base on Scrapyd, and added auth/filter/order…… and added new API
ScrapydArt is a service for running `Scrapy`_ spiders.
It allows you to deploy your Scrapy projects and control their spiders using an
HTTP JSON API.
The documentation (including installation and usage) can be found at(官方文档在此):
https://scrapydart.readthedocs.io/zh/latest/
.. image:: https://github.com/dequinns/ScrapydArt/blob/master/images/scrapydart-doc.png
.. _Scrapy: https://github.com/dequinns/scrapydart
ScrapydArt
==========
.. image:: https://secure.travis-ci.org/scrapy/scrapyd.svg?branch=master
:target: http://travis-ci.org/scrapy/scrapyd
.. image:: https://codecov.io/gh/scrapy/scrapyd/branch/master/graph/badge.svg
:target: https://codecov.io/gh/scrapy/scrapyd
==========
项目演示地址
==========
http://scrapydart.porters.vip
==========
拒绝裸奔
==========
ScrapydArt在Scrapyd基础上新增了权限验证、筛选过滤、排序、数据统计以及排行榜等功能,并且有了更强大的API。
scrapyd是世界最优秀的爬虫框架scrapy官方提供的部署控管理平台,它提供了爬虫任务调用的api、爬虫运行日志功能。
但它并不提供权限验证功能,意味着只要你部署到服务器,任何人都可以访问它并且控制你的爬虫。它也不提供数据统计的功能,你不可能知道你部署的爬虫有多少、也不知道它们运行了多少次、谁的运行时间最长、更别说排行了。官方开发人员关注的重点是api功能性而非界面,所以界面(甚至可以说没有界面)不谈美观。
==========
ScrapydArt安装
==========
pip install scrapydart
==========
ScrapydArt启动
==========
scrapydart
==========
ScrapydArt-web界面
==========
http://localhost:6800
--------------------------------------------------
更多使用方法和ScrapydArt的详细介绍请阅读ScrapydArt文档
--------------------------------------------------
https://scrapydart.readthedocs.io/zh/latest/
--------------------
下图为ScrapydArt界面
--------------------
.. image:: https://github.com/dequinns/ScrapydArt/blob/master/images/scrapydart-home.png
.. image:: https://github.com/dequinns/ScrapydArt/blob/master/images/scrapydart-jobs.png
ScrapydArt base on Scrapyd, and added auth/filter/order…… and added new API
ScrapydArt is a service for running `Scrapy`_ spiders.
It allows you to deploy your Scrapy projects and control their spiders using an
HTTP JSON API.
The documentation (including installation and usage) can be found at(官方文档在此):
https://scrapydart.readthedocs.io/zh/latest/
.. image:: https://github.com/dequinns/ScrapydArt/blob/master/images/scrapydart-doc.png
.. _Scrapy: https://github.com/dequinns/scrapydart
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapydart-1.2.0.327.tar.gz
(79.7 kB
view details)
File details
Details for the file scrapydart-1.2.0.327.tar.gz
.
File metadata
- Download URL: scrapydart-1.2.0.327.tar.gz
- Upload date:
- Size: 79.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.13.0 setuptools/39.1.0 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.6.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | dec09ad7dbf961f35afa755b90c7c3a2ba69e8e9452e5b25dbdd3391b51b3d32 |
|
MD5 | 2e5315dca48a8a5336bb451abafa7e08 |
|
BLAKE2b-256 | 0c0d9a81ad458e11a5fb45794838f2bedabbdbd0892d5e0e32b98d0eaa530aef |