A distributed crawler component based on scrapy_redis which can specify the expiration time of fingerprints in dupefilter.
Project description
The author of this package has not provided a project description
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for scrapy-redis-expiredupefilter-0.2.1.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | df05557d85f7263bed6645ce2d2eaf69dbcffaa6d501049d410b3eb00e177ba2 |
|
| MD5 | 4155613762e7739a3a4d4f021ce48b66 |
|
| BLAKE2b-256 | 443ff81b89a5c6bba62aef09f391a8841ff1edd4e04726837efa3ba1df7fe4cc |
