A distributed crawler component based on scrapy_redis which can specify the expiration time of fingerprints in dupefilter.
The author of this package has not provided a project description
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for scrapy-redis-expiredupefilter-0.2.1.tar.gz