A library to filter SQLAlchemy queries.
Project description
Scrapyq is designed to replace the SQLite backend by a Redis backend. In other words, all the queue management will be done using Redis.
Scrapyq is a fork of the original https://github.com/speakol-ads/scrapyd-redis implementation.
Install
pip install scrapyq
Config
To start using this library you just need to override the spiderqueue option in your scrapyd.conf file:
[scrapyd]
spiderqueue = scrapyq.spiderqueue.RedisSpiderQueue
...
If you want to customize the access to the database, you can add into your scrapyd.conf file:
[scrapyq]
queue_prefix = scrapyq.queue.
redis_db = 0
redis_host = localhost
redis_port = 6379
redis_username = 'admin' # (Optional)
redis_password = 'password' # (Optional)
...
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapyq-1.0.0.tar.gz
(3.9 kB
view hashes)