aiohttp client workers with redis in/out queues
Project description
Installation
$ pip install aiohttp-rq
Environment variables
Variable | default |
---|---|
AIOHTTP_RQ_DIR |
None |
AIOHTTP_RQ_REQUEST_QUEUE |
aiohttp-rq-request |
AIOHTTP_RQ_RESPONSE_QUEUE |
aiohttp-rq-response |
AIOHTTP_RQ_EXCEPTION_QUEUE |
aiohttp-rq-exception |
optional
Variable | default |
---|---|
AIOHTTP_RQ_DIR |
None |
AIOHTTP_RQ_REDIS_HOST |
localhost |
AIOHTTP_RQ_REDIS_PORT |
6379 |
AIOHTTP_RQ_REDIS_DB |
0 |
Examples
$ export AIOHTTP_RQ_REQUEST_QUEUE="aiohttp-rq-request"
$ export AIOHTTP_RQ_RESPONSE_QUEUE="aiohttp-rq-response"
$ export AIOHTTP_RQ_EXCEPTION_QUEUE="aiohttp-rq-exception"
$ python3 -m aiohttp_rq 100 # 100 workers
Redis
import redis
REDIS = redis.Redis(host='localhost', port=6379, db=0)
Redis push
value=json.dumps(dict(
url='https://domain.com',
method="GET",
headers=None,
data=None,
allow_redirects=True
))
REDIS.rpush('aiohttp-rq-request',*values)
Redis pull
item_list = REDIS.lrange('aiohttp-rq-response',0,-1)
for item in item_list:
data = json.loads(item.encode('utf-8'))
item_list = REDIS.lrange('aiohttp-rq-exception',0,-1)
for item in item_list:
data = json.loads(item.encode('utf-8'))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
aiohttp_rq-0.0.2.tar.gz
(3.2 kB
view hashes)