Skip to main content

Easy Sipder


Easy Spider 主要有四大模块:

  • Spider 负责推送请求到请求线程池
  • Downloader 负责启动请求与数据,请求在启动前会经过请求处理程序,响应在下载后会经过响应处理程序
  • Pipeline 负责清理数据,数据的持久化等工作

流程图如下

epsider流程图


TODO

  • 2020-04-06

    • 修复 start_requests 错误提示
    • 自动设置请求优先级
    • 请求和响应扩展合并为下载中间件
    • settings 像 scrapy 看齐
  • 2020-04-07

    • 优化 setting
    • 下载器开始停止问题

Metadata

Release files for espider 1.7.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for espider 1.7.3
File Size Uploaded Provenance Actions
espider-1.7.3.tar.gz 46.8 kB Without provenance Details for espider-1.7.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for espider 1.7.3
File Interpreter ABI Platform Actions
espider-1.7.3-py2.py3-none-any.whl Python 3, Python 2 none any Details for espider-1.7.3-py2.py3-none-any.whl

Total release size: 98.1 kB

Release files / espider-1.7.3.tar.gz

Download URL espider-1.7.3.tar.gz
Size 46.8 kB
Tags Source
SHA-256 checksum
How to use checksums
01b9f34e04517ddf9408d3716a1d3a5be085bf7197ed182a93cf6155a528e778
BLAKE2b-256 checksum
How to use checksums
5a6d81a313d085e9ad2289c1ec677193e8cd8b73a48203cba5fd334a730e1c28
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.2 importlib_metadata/4.6.3 pkginfo/1.7.1 requests/2.23.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.8.10

Release files / espider-1.7.3-py2.py3-none-any.whl

Download URL espider-1.7.3-py2.py3-none-any.whl
Size 51.2 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
dee32d19608f2ef185fda0ee06cbf9b789843f9a59a3fe5611764b66d2e177b8
BLAKE2b-256 checksum
How to use checksums
24ed65df191e67749749174c4a2a6f70efc0503b04a727126782e098973db1c4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.2 importlib_metadata/4.6.3 pkginfo/1.7.1 requests/2.23.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.8.10

Security for espider 1.7.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page