Skip to main content

Distributed Python web crawling framework

Project description

========
Dragline
========

[![PyPI version](https://badge.fury.io/py/Dragline.svg)](http://badge.fury.io/py/Dragline)

Dragline is a distributed Python web crawling framework.

Features :
---------
* Distributed, scalable and persistent crawling
* Efficient lightweight parallel execution based on gevent
* Redis backend for distributed and uninterrupted crawling


Requirements
------------

* Python 2.7
* Works on Linux
* Redis


Install
-------

The quick way :

pip install --pre dragline


Download the latest release from [Python Package Index] or clone [the repository].


[Python Package Index]: http://pypi.python.org/pypi/dragline
[the repository]: https://github.com/inzyte/dragline

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Dragline-2.0.0.tar.gz (16.6 kB view hashes)

Uploaded Source

Built Distribution

Dragline-2.0.0-py2-none-any.whl (22.3 kB view hashes)

Uploaded Python 2

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page