Skip to main content

No project description provided

Project description

redis-bulk-cleaner

Python PyPi image

Deletes keys from Redis database in bulk.

Features

  • Built for very large scale databases (10Gb+, 100 000 000+ keys)
  • Uses SCAN operation so it is safe to run it without downtime
  • Could process multiple patterns using only one SCAN operation
  • Has option to search for given patterns, but not delete them

Usage

Install it via PyPi

pip install redis_bulk_cleaner

Firstly check what are you going to delete with --dry-run option:

$ redis_bulk_cleaner 'test:unsubscribe_token:*' 'test:session:*' --dry-run
Search for keys:
test:unsubscribe_token:*, test:session:*
===========
test:unsubscribe_token:NDCivQ45KQcTghpS
test:session:NDCivQ45KQcTghpS
test:unsubscribe_token:9SBQ1YsDPuTETWBS
test:unsubscribe_token:MckwaZGER1GiVjoX
test:session:9SBQ1YsDPuTETWBS
test:session:MckwaZGER1GiVjoX

You could pass as many patterns as you want. Only overhead will be Regex matching slowdown.

Then to actually delete keys run it without --dry-run option:

$ redis_bulk_cleaner 'test:unsubscribe_token:*' 'test:session:*'
This tools will delete ANY key that matches any of the following pattern:
test:unsubscribe_token:*, test:session:*
Do you want to continue? (y, n): y
...
6 keys cleaned

Be ware: it uses regex matching so test* will also match test_other:key. However any special symbols like .? will be escaped.

asciicast

Note: because of SCAN behaviour it is not possible to calculate accurate estimate/percentage. Progress bar could go beyond 100%.

  • -D, --dry-run - Do not delete keys just print them
    • Default: disabled
  • -r, --use-regex - By default patterns are redis-patterns (* equals any characters sequence including :). If this flag is enabled then it will parse patterns as python regex expressions (some escaping may still be needed base on your shell type)
    • Default: disabled
  • -s, --sleep - (milliseconds) Sleep between DELETE commands to prevent high memory and cpu usage. Disabled by default.
  • -h, --host - Redis server host
    • Default: localhost
  • -p, --port - Redis server port
    • Default: 6379
  • --db - Redis server db
    • Default: 0
  • -P, --password - Redis server password
    • Default: No password
  • -b, '--batch' - Redis SCAN batch size
    • Too small value will cause slowdown, too high value could lead to memory issues/timeouts
    • Default: 500
  • --disable-cursor-backups
    • By default current scan position is saving to temporary redis variable (redis_cleaner:cursor) so it will continue from same place after restart (in case of crash/network issues/etc)
    • Default: Enabled

Also see redis_bulk_cleaner --help

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.


Free software: MIT license

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

redis_bulk_cleaner-0.1.5.tar.gz (9.0 kB view details)

Uploaded Source

Built Distribution

redis_bulk_cleaner-0.1.5-py2.py3-none-any.whl (7.2 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file redis_bulk_cleaner-0.1.5.tar.gz.

File metadata

  • Download URL: redis_bulk_cleaner-0.1.5.tar.gz
  • Upload date:
  • Size: 9.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/54.0.0 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7

File hashes

Hashes for redis_bulk_cleaner-0.1.5.tar.gz
Algorithm Hash digest
SHA256 f7c3ae3ba4e3c00500042f15e253d1a833b9137321925d804816d0ee13df4edb
MD5 edb6b29087faaeb4f586b6fd3943d738
BLAKE2b-256 f9a6365e1e4e3e51b0fc55fe0f870974fdea6fdfad7a022717218b90ae7784ba

See more details on using hashes here.

File details

Details for the file redis_bulk_cleaner-0.1.5-py2.py3-none-any.whl.

File metadata

  • Download URL: redis_bulk_cleaner-0.1.5-py2.py3-none-any.whl
  • Upload date:
  • Size: 7.2 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/54.0.0 requests-toolbelt/0.9.1 tqdm/4.58.0 CPython/3.8.7

File hashes

Hashes for redis_bulk_cleaner-0.1.5-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 ff4c9d2bc53c9efb1cf4d95fa7a6367b61b0c85e84399c94ec1e983a9963f827
MD5 988d779902c8cb3c7275d106af7c5482
BLAKE2b-256 262441c5e4d54693a3b693a5ee25b7551b25863382148b1ae0fea845b6a96f69

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page