Distributed reader-writer lock for python using redis as server
Features:
Reader-writer lock (can have multiple readers or one exclusive writer)
Stale locks collected (run as separate process, python3 -m redisrwlock)
Deadlock detection
Note: Deadlock detection and garbage/staleness collection is done in client side, which can cause excessive I/O with redis server. Tune with retry_interval and consider running the stale lock collection appropriately for your purpose.
Dependencies:
python 3.5.2
redis-py 2.10.5
redis 3.2.6
[test] Coverage.py 4.2
Install
pip install redisrwlock
Usages
Try lock with timeout=0
With timeout=0, RwlockClinet.lock acts as so called try_lock.
from redisrwlock import Rwlock, RwlockClient
client = RwlockClient()
rwlock = client.lock('N1', Rwlock.READ, timeout=0)
if rwlock.status == Rwlock.OK:
# Processings of resource named 'N1' with READ lock
# ...
client.unlock(rwlock)
elif rwlock.status == Rwlock.FAIL:
# Retry locking or quit
Waiting until lock success or deadlock
With timout > 0, RwlockClient.lock waits until lock successfully or deadlock detected and caller is chosen as victim.
from redisrwlock import Rwlock, RwlockClient
client = RwlockClient()
rwlock = client.lock('N1', Rwlock.READ, timeout=Rwlock.FOREVER)
if rwlock.status == Rwlock.OK:
# Processings of resource named 'N1' with READ lock
# ...
client.unlock(rwlock)
elif rwlock.status == Rwlock.DEADLOCK:
# 1. unlock if holding any other locks
# 2. Retry locking or quit
Removing stale locks
When a client exits without unlock, redis keys for the client’s locks remain in server and block other clients from successful locking. redisrwlock run in command line removes such garbage locks, waits in server.
python3 -m redisrwlock
You can repeat this gc periodically by specifying -r or –repeat option.
Tests
Unittest
Runnig unittest in test directory:
cd test python3 -m unittest -qor in project top directory:
python3 -m unittest discover test -q
Examples below are assuming you run unittest in project top directory.
Coverage
coverage erase
coverage run -a -m unittest discover test -q
coverage html
Above simple coverage run will report lower coverage than expected because the tests use subprocess. Codes run by subprocess are not covered in report by default.
Subprocess coverage
Need some preperation:
Edit sitecustomize.py (under python intallation’s site-packages directory), add 2 lines
import coverage coverage.process_startup()Edit .coveragerc (default name of coverage.py’s config file)
[run] branch = True [html] directory = htmlcov
Then, run coverage with environment variable COVERAGE_PROCESS_START={path/to/coveragerc}
coverage erase
COVERAGE_PROCESS_START=.coveragerc coverage run -a -m unittest discover test -q
coverage html
TODOs
TODO: command line option to specify redis-server
TODO: high availability! redis sentinel or replication?
Metadata
Release files for redisrwlock 0.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| redisrwlock-0.1.2.tar.gz | 11.7 kB | Without provenance | Details for redisrwlock-0.1.2.tar.gz |
Release files / redisrwlock-0.1.2.tar.gz
| Download URL | redisrwlock-0.1.2.tar.gz |
|---|---|
| Size | 11.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
350ceb0b4483738224cfbdea258a649e7b0670b710da6e9ceebea7ae26f82873
|
|
BLAKE2b-256 checksum How to use checksums |
2b6906152288a240d2fd6e840f240095c1884bfed26da499fbfd876656ae22f7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for redisrwlock 0.1.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.