ScrapyCloud HubStorage frontier backend for Frontera
Project description
When used with scrapy, use it with Scrapy Scheduler provided by scrapy-frontera. Scrapy scheduler provided by Frontera is not supported
See usage instructions at module and class docstrings at backend.py.
Package also installs a convenient command line tool for hubstorage frontier handling and manipulation: hcfpal.py. It supports dumping, count, deletion, moving, listing, etc.
Another tool provided is hcfmanager.py. It facilitates the scheduling of consumer spider jobs.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hcf-backend-0.2.4.1.tar.gz
(10.4 kB
view hashes)
Built Distribution
Close
Hashes for hcf_backend-0.2.4.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c9bf67bfb82d4c40998eb78faf42ff36ad331d3861eaa4c39d6dde2fdbdaaef3 |
|
MD5 | 5ae6c95f1ca672e20c4c6c0ce711244c |
|
BLAKE2b-256 | cd3f6cac7b8580279245e8a7be3d8118100f1e0a917ec04be62e378f8b156dd2 |