Kafka-based components for Scrapy
Kafka-based components for Scrapy, to allow Spider`s to read their next URL from a Kafka topic and an `ItemPipeline to store the JSON-ified Item back into a Kafka topic.
Please see the example directory for how to use this.
Contributors to scrapy-kafka, listed alphabetically:
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.