Skip to main content

A simple async Python module to bypass Cloudflare's anti-bot page. See https://github.com/pavlodvornikov/aiocfscrape for more information.

Project description

A simple async Python module to bypass Cloudflare’s anti-bot page. Based on aiohttp ClientSession. Solution was inherited from cfscrape module.

You could use it eg. with Python 3 and asyncio for concurrent crawling of web resources protected with CloudFlare.

Installation

Install with pip

pip install aiocfscrape

Basic Usage

aiocfscrape is a aiohttp.ClientSession wrapper. So aiohttp client reference can be used as the base.

To make simple get request do the following:

import asyncio
from aiocfscrape import CloudflareScraper

async def test_open_page(url):
    async with CloudflareScraper() as session:
        async with session.get(url) as resp:
            return await resp.text()

if __name__ == '__main__':
    asyncio.run(test_open_page('<your url>'))

Dependencies

License

aiocfscrape is offered under the MIT license.

Changelog

1.0.0 (2020-03-25)

0.0.9 (2019-03-21)

  • Update anit-bot integration to latest cfscrape version [gaardiolor]

0.0.8 (2019-03-16)

  • Update anit-bot integration to latest cfscrape version [gaardiolor]

0.0.7 (2019-01-10)

  • Fix error when host does not return a “Server” header [pavlodvornikov]
  • Update anit-bot integration to latest cfscrape version [gaardiolor]

0.0.6 (2018-06-19)

  • Update readme [pavlodvornikov]

0.0.5 (2018-06-19)

  • Fix aiohttp version 3 compatibility [ape364]
  • Bump js2py [ape364]
  • Update anti-bot integration to latest cfscrape version [dteh]

0.0.4 (2018-03-06)

  • Update anti-bot integration to latest cfscrape version [slazarov]

0.0.3 (2017-11-21)

  • Fix AttributError [ape364]

0.0.2 (2016-08-26)

  • Support timeout inside request [pavlodvornikov]
  • Update package information [pavlodvornikov]
  • Remove obsolete files [pavlodvornikov]

0.0.1 (2016-08-26)

  • Initial implementation

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for aiocfscrape, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size aiocfscrape-1.0.0.tar.gz (6.9 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page