HSSP 爬虫框架
一个基于python asyncio开发的爬虫框架 (开发中)
作者
特性
- 使用scrapy框架的选择器 parsel 作为内置网页选择器
- 基于tenacity的自动异常重试
- 基于fake-useragent的可选随机UA
- 可选的多种下载器: httpx、aiohttp、requests、curl-cffi、requests-go
- 请求前、响应后、重试后监听
路线
- 增加其他解析器
- 在情求过程中临时更换下载器：比如net初始化时使用的是httpx下载器，其中一个情求要临时切换至
DrissionPage, 其他的依旧是httpx - 支持
DrissionPage、playwright浏览器渲染的下载器 - 下载器支持更多配置项及自定义项
- 编写详细使用文档
安装
使用 pip 安装 hssp
pip install hssp
使用 uv 安装 hssp
uv add hssp
支持
如需支持，请发送电子邮件至 xhrtxh@gmail.com。
开发测试
项目使用uv管理依赖，需先安装uv
rye sync
1 maintainer
Metadata
1 maintainer
Release files for hssp 0.4.18
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| hssp-0.4.18.tar.gz | 17.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| hssp-0.4.18-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 40.0 kB
Release files / hssp-0.4.18.tar.gz
| Download URL | hssp-0.4.18.tar.gz |
|---|---|
| Size | 17.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5032ecfb90d8a40476080da435b8d1934c66490296252bd9d83ec685d7271fdf
|
|
BLAKE2b-256 checksum How to use checksums |
1b49bcd2d8e6e79b53e8dd89882e0d0763d252c58cae7f724a8b353836ad4da7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.8
|
Release files / hssp-0.4.18-py3-none-any.whl
| Download URL | hssp-0.4.18-py3-none-any.whl |
|---|---|
| Size | 23.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
95c7fe09d1e1611c11bd73a80093fa4945dd8188b015e1c0c8ef28b903799517
|
|
BLAKE2b-256 checksum How to use checksums |
cdc0e9ecead4df168d3c4db493cb00fa3fb6591bb388ce805e366ef72e67d1e0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.8
|
