scrapy 常用爬网必备工具包
Project description
Scrapy+
Scrapy扩展工具包。为《从0学爬虫专栏》 提供，详细的使用方法请到专栏内参考。
$ pip install scrapy_plus
Scrapy+提供以下的内容
- 过滤器
- Redis 去重过滤器
- Redis 布隆去重过滤器
- 中间件
- 自登录中间件
- 花瓣网专用中间件
- Chrome通用中间件
- Splash渲染中间件
- Tor中间件
- 随机UA中间件
- 随机代理中间件
- 管道
- MongoDB数据存储管道
- 可支持阿里云的OSS图片管道
- SQL存储端
- 输入/输出处理器
- 蜘蛛
BookSpiderNeteaseSpiderTaobaoSpider
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapy_plus-1.0.5.tar.gz
(21.0 kB
view details)
Built Distribution
File details
Details for the file scrapy_plus-1.0.5.tar.gz.
File metadata
- Download URL: scrapy_plus-1.0.5.tar.gz
- Upload date:
- Size: 21.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.25.1 setuptools/49.6.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.8.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 4cff41960e017453df7184643bcdfd813eae3d2bbd826d0be29eeb63cf7e22f6 |
|
| MD5 | dd5c96c00854fbee2745f01056c05e3c |
|
| BLAKE2b-256 | 301fdc3d7f1ed3a3768e171bed9c6bc6b4befa7b69b171804196e0541e9d925a |
File details
Details for the file scrapy_plus-1.0.5-py3-none-any.whl.
File metadata
- Download URL: scrapy_plus-1.0.5-py3-none-any.whl
- Upload date:
- Size: 29.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.25.1 setuptools/49.6.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.8.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | e4cb29ba0845bd4717c2c5f66ba31affd8154c09438a2dcc20c5975c442c952c |
|
| MD5 | c80384abb0367812cc872df941d5978f |
|
| BLAKE2b-256 | ceab8ff5db0cf85a46e581c3d5086ef0510225712e41e15cd52af1e0171e065e |
