Skip to main content

A Scrapy Middleware to bypass the CloudFlare's anti-bot protection

Project description


## Scrapy "CloudFlare" middleware

A Scrapy Middleware to bypass the CloudFlare's anti-bot protection, based on [cloudflare-scrape](

### Installation
pip install scrapy_cloudflare_middleware

### Usage

Add the middleware to your `DOWNLOADER_MIDDLEWARES` settings

'scraping_hub.middlewares.CloudFlareMiddleware': 560
The priority (560) is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.

Done. Happy scraping !

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for scrapy-cloudflare-middleware, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size scrapy_cloudflare_middleware-0.0.1-py3-none-any.whl (3.8 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size scrapy_cloudflare_middleware-0.0.1.tar.gz (2.1 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page