scrapy-cloudflare-middleware

A Scrapy Middleware to bypass the CloudFlare's anti-bot protection

Project description

[![PyPI](https://img.shields.io/pypi/v/scrapy_cloudflare_middleware.svg)](https://pypi.python.org/pypi/scrapy_cloudflare_middleware)

## Scrapy "CloudFlare" middleware

A Scrapy Middleware to bypass the CloudFlare's anti-bot protection, based on [cloudflare-scrape](https://github.com/Anorov/cloudflare-scrape).

### Installation
```
pip install scrapy_cloudflare_middleware
```

### Usage

Add the middleware to your `DOWNLOADER_MIDDLEWARES` settings

```python
DOWNLOADER_MIDDLEWARES = {
'scraping_hub.middlewares.CloudFlareMiddleware': 560
}
```
The priority (560) is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.

Done. Happy scraping !

Project details

Release history Release notifications | RSS feed

This version

0.0.1

Feb 10, 2018

0.0.1.dev1 pre-release

Feb 10, 2018

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy_cloudflare_middleware-0.0.1.tar.gz (2.1 kB view hashes)

Uploaded Feb 10, 2018 Source

Built Distribution

scrapy_cloudflare_middleware-0.0.1-py3-none-any.whl (3.8 kB view hashes)

Uploaded Feb 10, 2018 Python 3

Hashes for scrapy_cloudflare_middleware-0.0.1.tar.gz

Hashes for scrapy_cloudflare_middleware-0.0.1.tar.gz
Algorithm	Hash digest
SHA256	`c2157224c6e5df0ec8f572031f822ef696fbf4e06e45f78841bfd74ebdf58d58`
MD5	`f224bda6f0b981aee81d363d890f93c7`
BLAKE2b-256	`40a9318f4d04af28560ec676989ba419d4e43e5908a90fd5bd6a236b9ec5e178`

Hashes for scrapy_cloudflare_middleware-0.0.1-py3-none-any.whl

Hashes for scrapy_cloudflare_middleware-0.0.1-py3-none-any.whl
Algorithm	Hash digest
SHA256	`a288e419a6080361beec49093f6f4107cb43ddb252898dbc24ad8ce2941a21a1`
MD5	`d1fee458a5d62db3f4460a9fe34e68f7`
BLAKE2b-256	`3f0c2b8f895012ae01ec40b780b4c100427fd002239b000bbe3833685e05c8f2`