A Simple Web Crawling and Web Scraping framework
Project description
Crwy
简介
Crwy是一个轻量级的爬虫抓取框架,参考Scrapy框架结构开发而来。该框架提供了实用的爬虫模板,旨在帮助大家快速实现爬虫任务,高效开发。并为scrapy使用者提供通用轮子.。新增了gevent,使爬虫异步执行,速度更快。
运行环境
Python2 & Python3
Works on Linux, Mac OSX
安装
快速安装
pip install crwy
or 前往下载: https://pypi.python.org/pypi/Crwy/
TODO
完善scrapy_plugs
完善selenium_api
兼容python3
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Crwy-1.6.0.tar.gz
(27.5 kB
view hashes)
Built Distributions
Crwy-1.6.0-py3.6.egg
(99.4 kB
view hashes)
Crwy-1.6.0-py2.7.egg
(97.6 kB
view hashes)