redis_canal
Project description
redis-canal
Proxy redis stream from one to another through global queue service.
Motivation: Redis provided on the cloud is usually only available within a VPC and does not provide external access. We want to provide a way to synchronize messages across clouds without going through a VPN.
Supported queue service
- AWS SQS
Welcome to contribute more queue service, see adapter/impl for more details.
We also support the plugin system, the adapter can also be imported as a third-party library. See example for more details.
Install
pip install redis-canal[all]
for all components.
Or use docker image
docker pull wh1isper/redis-canal
Usage
Use CLI
redis-canal stream-to-queue
to start a daemon to consume redis stream and push to global queueredis-canal queue-to-stream
to pull from global queue and push to redis stream
CLI args support envvar
, which you can refer to envs.py.
TODO
- More adapter for Google PubSub, Kafka, etc
- Allows users to more easily configure the adapter(Now only support environment variables, within
Adapter
)
Develop
Install pre-commit before commit
pip install pre-commit
pre-commit install
Install package locally
pip install -e .[test]
Run unit-test before PR
pytest -v tests
Develop sqs adapter
You need to configure your AWS account first. See AWS CLI docs for more details.
Make sure your account has sqs:*
permission. Then you can run pytest -v tests/adapter/test_sqs_adapter.py
to test it.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for redis_canal-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 048ad50f7a6e4d5fc98d9b1f86adc56a5236cfe4b3eaf12348fcf3d577c7ee7a |
|
MD5 | 92f0131abaf3d99edbc15a46cc75e069 |
|
BLAKE2b-256 | 2f7043cb065fabfcc28c966e2d4c8f20f59cbd08eafed4a39fe8118dc6454c2c |