ws 0.1

Efficient asyncio based web crawler

Features

• Asynchronous downloading using aiohttp

• All downloads cached locally in sqlite

• Continue an interrupted crawl

Example

>>> import ws
...

Install

Install from pypi:

pip install ws

Or checkout latest version from repository:

hg clone https://bitbucket.org/richardpenman/ws