Skip to main content
qCrawl Logo

PyPI Version PyPI - Python Version Codecov

qcrawl is a fast async web crawling & scraping framework for Python to extract structured data from web-pages. It is cross-platform and easy to install via pip or conda.

Follow the documentation.

qCrawl features

  1. Async architecture - High-performance concurrent crawling based on asyncio
  2. Performance optimized - Queue backend on Redis with direct delivery, messagepack serialization, connection pooling, DNS caching
  3. Powerful parsing - CSS/XPath selectors with lxml
  4. Middleware system - Customizable request/response processing
  5. Flexible export - Multiple output formats including JSON, CSV, XML
  6. Flexible queue backends - Memory, Disk, Redis-based schedulers for different scale requirements
  7. Item pipelines - Data transformation, validation, and processing pipeline
  8. Pluggable downloaders - HTTP (aiohttp), Camoufox (stealth browser) for JavaScript rendering and anti-bot evasion

Metadata

Release files for qcrawl 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for qcrawl 0.4.0
File Size Uploaded Provenance Actions
qcrawl-0.4.0.tar.gz 247.6 kB Without provenance Details for qcrawl-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for qcrawl 0.4.0
File Interpreter ABI Platform Actions
qcrawl-0.4.0-py3-none-any.whl Python 3 none any Details for qcrawl-0.4.0-py3-none-any.whl

Total release size: 374.6 kB

Release files / qcrawl-0.4.0.tar.gz

Download URL qcrawl-0.4.0.tar.gz
Size 247.6 kB
Tags Source
SHA-256 checksum
How to use checksums
a1c58d4948d3d373f7df44e59ea9c79a9ee321e78f15af4914653dd5343130f3
BLAKE2b-256 checksum
How to use checksums
5464066ff131a67962bfa9ace0d698ff353eef410b1e969d93bf154fd20d6c89
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.5

Release files / qcrawl-0.4.0-py3-none-any.whl

Download URL qcrawl-0.4.0-py3-none-any.whl
Size 127.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
dfa1007821bbcfbd2452efe00c9e2bb7f53559a2fe322c798923106de51af3d8
BLAKE2b-256 checksum
How to use checksums
2ecdce47ad5cfd14b274b312ea60c410e3458e272e1c4dce5c4497c648f90785
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.5

Security for qcrawl 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page