No project description provided
Project description
redis-bulk-cleaner
Deletes keys from Redis database in bulk.
Features
- Built for very large scale databases (10Gb+, 100 000 000+ keys)
- Uses SCAN operation so it is safe to run it without downtime
- Could process multiple patterns using only one SCAN operation
- Has option to search for given patterns, but not delete them
Usage
Install it via PyPi
pip install redis_bulk_cleaner
Firstly check what are you going to delete with --dry-run
option:
$ redis_bulk_cleaner 'test:unsubscribe_token:*' 'test:session:*' --dry-run
Search for keys:
test:unsubscribe_token:*, test:session:*
===========
test:unsubscribe_token:NDCivQ45KQcTghpS
test:session:NDCivQ45KQcTghpS
test:unsubscribe_token:9SBQ1YsDPuTETWBS
test:unsubscribe_token:MckwaZGER1GiVjoX
test:session:9SBQ1YsDPuTETWBS
test:session:MckwaZGER1GiVjoX
You could pass as many patterns as you want. Only overhead will be Regex matching slowdown.
Then to actually delete keys run it without --dry-run
option:
$ redis_bulk_cleaner 'test:unsubscribe_token:*' 'test:session:*'
This tools will delete ANY key that matches any of the following pattern:
test:unsubscribe_token:*, test:session:*
Do you want to continue? (y, n): y
...
6 keys cleaned
Be ware: it uses regex matching so test*
will also match test_other:key
.
However any special symbols like .?
will be escaped.
Note: because of SCAN behaviour it is not possible to calculate accurate estimate/percentage. Progress bar could go beyond 100%.
-D
,--dry-run
- Do not delete keys just print them- Default: disabled
-r
,--use-regex
- By default patterns are redis-patterns (*
equals any characters sequence including:
). If this flag is enabled then it will parse patterns as python regex expressions (some escaping may still be needed base on your shell type)- Default: disabled
-s
,--sleep
- (milliseconds) Sleep between DELETE commands to prevent high memory and cpu usage. Disabled by default.-h
,--host
- Redis server host- Default:
localhost
- Default:
-p
,--port
- Redis server port- Default:
6379
- Default:
--db
- Redis server db- Default:
0
- Default:
-P
,--password
- Redis server password- Default: No password
-b
, '--batch' - Redis SCAN batch size- Too small value will cause slowdown, too high value could lead to memory issues/timeouts
- Default:
500
--disable-cursor-backups
- By default current scan position is saving to temporary redis variable (
redis_cleaner:cursor
) so it will continue from same place after restart (in case of crash/network issues/etc) - Default: Enabled
- By default current scan position is saving to temporary redis variable (
Also see redis_bulk_cleaner --help
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
Free software: MIT license
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file redis_bulk_cleaner-0.1.5.tar.gz
.
File metadata
- Download URL: redis_bulk_cleaner-0.1.5.tar.gz
- Upload date:
- Size: 9.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/54.0.0 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7c3ae3ba4e3c00500042f15e253d1a833b9137321925d804816d0ee13df4edb |
|
MD5 | edb6b29087faaeb4f586b6fd3943d738 |
|
BLAKE2b-256 | f9a6365e1e4e3e51b0fc55fe0f870974fdea6fdfad7a022717218b90ae7784ba |
File details
Details for the file redis_bulk_cleaner-0.1.5-py2.py3-none-any.whl
.
File metadata
- Download URL: redis_bulk_cleaner-0.1.5-py2.py3-none-any.whl
- Upload date:
- Size: 7.2 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/54.0.0 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ff4c9d2bc53c9efb1cf4d95fa7a6367b61b0c85e84399c94ec1e983a9963f827 |
|
MD5 | 988d779902c8cb3c7275d106af7c5482 |
|
BLAKE2b-256 | 262441c5e4d54693a3b693a5ee25b7551b25863382148b1ae0fea845b6a96f69 |