一个简单快速的异步爬虫框架
Project description
HSSP 爬虫框架
一个基于python asyncio开发的爬虫框架 (开发中)
作者
特性
- 使用scrapy框架的选择器
parsel作为内置网页选择器 - 基于tenacity的自动异常重试
- 基于fake-useragent的可选随机UA
- 可选的多种下载器: httpx、aiohttp、requests等
- 请求前、响应后、重试后监听
安装
使用 pip 安装 hssp
pip install hssp
使用 rye 安装 hssp
rye add hssp
路线图
- 基于fake-useragent的随机UA
- curl-cff的支持
- drissionpage的支持
支持
如需支持，请发送电子邮件至 xhrtxh@gmail.com。
开发测试
项目使用rye管理依赖，需先安装rye
rye sync
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hssp-0.4.6.tar.gz
(18.3 kB
view details)
Built Distribution
hssp-0.4.6-py3-none-any.whl
(19.7 kB
view details)
File details
Details for the file hssp-0.4.6.tar.gz.
File metadata
- Download URL: hssp-0.4.6.tar.gz
- Upload date:
- Size: 18.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | e3f7f86e4579b5bd4b42966f6233acce3a7a6c464089bc41844396629b852123 |
|
| MD5 | c3eece24259c6da5914d63d96d26ffaa |
|
| BLAKE2b-256 | 81c9df5934326a704e7e76dac9ec299f1a20708210c268467edf58ea75182395 |
File details
Details for the file hssp-0.4.6-py3-none-any.whl.
File metadata
- Download URL: hssp-0.4.6-py3-none-any.whl
- Upload date:
- Size: 19.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b43186168253750cb881dde94a0c3435a324431b124d99ce94d340e77b46b08c |
|
| MD5 | b33ceef6fd0b748f16b8f737408e6fb3 |
|
| BLAKE2b-256 | f4789c695f1ab8081fa759d3f59a1672604aecb9b8e884e99f461cd29cc87e4b |
