scrapy 常用爬网必备工具包
Project description
Scrapy+
Scrapy扩展工具包。为《从0学爬虫专栏》 提供，详细的使用方法请到专栏内参考。
$ pip install scrapy_plus
Scrapy+提供以下的内容
- 过滤器
- Redis 去重过滤器
- Redis 布隆去重过滤器
- 中间件
- 自登录中间件
- 花瓣网专用中间件
- Chrome通用中间件
- Splash渲染中间件
- Tor中间件
- 随机UA中间件
- 随机代理中间件
- 管道
- MongoDB数据存储管道
- 可支持阿里云的OSS图片管道
- SQL存储端
- 输入/输出处理器
- 蜘蛛
BookSpiderNeteaseSpiderTaobaoSpider
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapy_plus-1.0.4.tar.gz
(19.6 kB
view details)
Built Distribution
File details
Details for the file scrapy_plus-1.0.4.tar.gz.
File metadata
- Download URL: scrapy_plus-1.0.4.tar.gz
- Upload date:
- Size: 19.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 5ae7b84a96a420956304fe5fa6c985030b28cb7224867fa0c9b167682c161c18 |
|
| MD5 | d16762e6fe6a0cb1ecd7a36eb607883d |
|
| BLAKE2b-256 | 156781ceca72ae038b429c4d3cb88d83d5b00c59351cc686e7d6274254fcaf77 |
File details
Details for the file scrapy_plus-1.0.4-py3-none-any.whl.
File metadata
- Download URL: scrapy_plus-1.0.4-py3-none-any.whl
- Upload date:
- Size: 28.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | fea3a5144392e412eb5cf0117bc24e4fda716163d97fd60787befa48549ec026 |
|
| MD5 | 73c488c2c9e6635c3b33190106c30ec2 |
|
| BLAKE2b-256 | 3eaf74d956f0d7395dce0b167a4434d62fe32b8df65879aa873788a4a200bd38 |
