Skip to main content

A Scrapy Middleware to bypass the CloudFlare's anti-bot protection

Project description

[![PyPI](https://img.shields.io/pypi/v/scrapy_cloudflare_middleware.svg)](https://pypi.python.org/pypi/scrapy_cloudflare_middleware)

## Scrapy "CloudFlare" middleware

A Scrapy Middleware to bypass the CloudFlare's anti-bot protection, based on [cloudflare-scrape](https://github.com/Anorov/cloudflare-scrape).

### Installation
```
pip install scrapy_cloudflare_middleware
```

### Usage

Add the middleware to your `DOWNLOADER_MIDDLEWARES` settings

```python
DOWNLOADER_MIDDLEWARES = {
'scraping_hub.middlewares.CloudFlareMiddleware': 560
}
```
The priority (560) is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.

Done. Happy scraping !








Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy_cloudflare_middleware-0.0.1.tar.gz (2.1 kB view details)

Uploaded Source

Built Distribution

File details

Details for the file scrapy_cloudflare_middleware-0.0.1.tar.gz.

File metadata

File hashes

Hashes for scrapy_cloudflare_middleware-0.0.1.tar.gz
Algorithm Hash digest
SHA256 c2157224c6e5df0ec8f572031f822ef696fbf4e06e45f78841bfd74ebdf58d58
MD5 f224bda6f0b981aee81d363d890f93c7
BLAKE2b-256 40a9318f4d04af28560ec676989ba419d4e43e5908a90fd5bd6a236b9ec5e178

See more details on using hashes here.

File details

Details for the file scrapy_cloudflare_middleware-0.0.1-py3-none-any.whl.

File metadata

File hashes

Hashes for scrapy_cloudflare_middleware-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 a288e419a6080361beec49093f6f4107cb43ddb252898dbc24ad8ce2941a21a1
MD5 d1fee458a5d62db3f4460a9fe34e68f7
BLAKE2b-256 3f0c2b8f895012ae01ec40b780b4c100427fd002239b000bbe3833685e05c8f2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page