A distributed crawler component based on scrapy_redis which can specify the expiration time of fingerprints in dupefilter.
Project description
The author of this package has not provided a project description
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for scrapy-redis-expiredupefilter-0.1.98.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d780470e5abca1a14199deb374a7a0946ce3d14253f239a7aa0de1e3b6721edf |
|
MD5 | aecc701928e523584f51a76c1717f4e6 |
|
BLAKE2b-256 | f52ed3b3d1b4fb5206df1cd3bd1f36baaaaa9f8e307037b815b96053c511a5c6 |