A Scrapy Middleware to bypass the CloudFlare's anti-bot protection
Project description
[![PyPI](https://img.shields.io/pypi/v/scrapy_cloudflare_middleware.svg)](https://pypi.python.org/pypi/scrapy_cloudflare_middleware)
## Scrapy "CloudFlare" middleware
A Scrapy Middleware to bypass the CloudFlare's anti-bot protection, based on [cloudflare-scrape](https://github.com/Anorov/cloudflare-scrape).
### Installation
```
pip install scrapy_cloudflare_middleware
```
### Usage
Add the middleware to your `DOWNLOADER_MIDDLEWARES` settings
```python
DOWNLOADER_MIDDLEWARES = {
'scraping_hub.middlewares.CloudFlareMiddleware': 560
}
```
The priority (560) is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.
Done. Happy scraping !
## Scrapy "CloudFlare" middleware
A Scrapy Middleware to bypass the CloudFlare's anti-bot protection, based on [cloudflare-scrape](https://github.com/Anorov/cloudflare-scrape).
### Installation
```
pip install scrapy_cloudflare_middleware
```
### Usage
Add the middleware to your `DOWNLOADER_MIDDLEWARES` settings
```python
DOWNLOADER_MIDDLEWARES = {
'scraping_hub.middlewares.CloudFlareMiddleware': 560
}
```
The priority (560) is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.
Done. Happy scraping !
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapy_cloudflare_middleware-0.0.1.dev1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ac0f4e8d55a621d3ae03d0c7191199a1841153848fc14f738a83b85c5a0661c |
|
MD5 | b8bd2084eca74e9785cca2742ea46efe |
|
BLAKE2b-256 | c486d7e581a5ee200b86fdbea3e32c19b9bbcf415cfaac5ce61f74d9cdf234b7 |
Close
Hashes for scrapy_cloudflare_middleware-0.0.1.dev1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | beb59a3b4a1eb1ab214621efabc8c854729231c263251b855b923783cc18300c |
|
MD5 | ea3d7feca80a632a06f55006af344cda |
|
BLAKE2b-256 | 3c14213f66c180300e4a6e8bc90b213bcbfe17e9114ad82a8ed516a9f335b049 |