一个简单快速的异步爬虫框架
Project description
HSSP 爬虫框架
一个基于python asyncio开发的爬虫框架 (开发中)
作者
特性
- 使用scrapy框架的选择器
parsel作为内置网页选择器 - 基于tenacity的自动异常重试
- 基于fake-useragent的可选随机UA
- 可选的多种下载器: httpx、aiohttp、requests等
- 请求前、响应后、重试后监听
安装
使用 pip 安装 hssp
pip install hssp
使用 rye 安装 hssp
rye add hssp
路线图
- 基于fake-useragent的随机UA
- curl-cff的支持
- drissionpage的支持
支持
如需支持，请发送电子邮件至 xhrtxh@gmail.com。
开发测试
项目使用rye管理依赖，需先安装rye
rye sync
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hssp-0.4.5.tar.gz
(18.3 kB
view details)
Built Distribution
hssp-0.4.5-py3-none-any.whl
(19.9 kB
view details)
File details
Details for the file hssp-0.4.5.tar.gz.
File metadata
- Download URL: hssp-0.4.5.tar.gz
- Upload date:
- Size: 18.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 3fb942863a437e6bcf8e6de019450abc9ad95fd0f6301c3e7b76be4a9aa721d5 |
|
| MD5 | 9333f1b4195eb60cb1b4994b8848d211 |
|
| BLAKE2b-256 | 39b7488921e4ad0d1339c920a9cffbce593b6e19566f6f02d59c9bd85ea226ac |
File details
Details for the file hssp-0.4.5-py3-none-any.whl.
File metadata
- Download URL: hssp-0.4.5-py3-none-any.whl
- Upload date:
- Size: 19.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ccdaae5036c5a4e333b6ceabc125171d5cd05015308b98c739e8275e777798f8 |
|
| MD5 | a1357507b27e101ad07801ff24eb237f |
|
| BLAKE2b-256 | d755b004dd8538785bfeeb1a7a27e4931dc8444340d4ea036e3f5a21362c0d27 |
