ScrapyCloud HubStorage frontier backend for Frontera
Project description
When used with scrapy, use it with Scrapy Scheduler provided by scrapy-frontera. Scrapy scheduler provided by Frontera is not supported
See usage instructions at module and class docstrings at backend.py.
Package also installs a convenient command line tool for hubstorage frontier handling and manipulation: hcfpal.py. It supports dumping, count, deletion, moving, listing, etc.
Another tool provided is hcfmanager.py. It facilitates the scheduling of consumer spider jobs.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hcf-backend-0.2.5.5.tar.gz
(10.3 kB
view hashes)
Built Distribution
Close
Hashes for hcf_backend-0.2.5.5-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | d85a956977013483a799393798ecd1fc87ceaa1113554d280db8e75f64308c76 |
|
| MD5 | c8f004f02d58d46e80226d9d0f54d977 |
|
| BLAKE2b-256 | 4122871d4adb92a5c7b2f06b8b712318d3cbe056392b92a0330390d774aee570 |
