Distributed inter-process locks with a choice of backend.
Sherlock is a library that provides easy-to-use distributed inter-process locks and also allows you to choose a backend of your choice for lock synchronization.
When you are working with resources which are accessed by multiple services or distributed services, more than often you need some kind of locking mechanism to make it possible to access some resources at a time.
Distributed Locks or Mutexes can help you with this. Sherlock provides the exact same facility, with some extra goodies. It provides an easy-to-use API that resembles standard library’s threading.Lock semantics.
Apart from this, Sherlock gives you the flexibility of using a backend of your choice for managing locks.
Sherlock also makes it simple for you to extend Sherlock to use backends that are not supported.
- API similar to standard library’s threading.Lock.
- Support for With statement, to cleanly acquire and release locks.
- Backend agnostic: supports Redis, Memcached and Etcd as choice of backends.
- Extendable: can be easily extended to work with any other of backend of choice by extending base lock class. Read extending.
Supported Backends and Client Libraries
Following client libraries are supported for every supported backend:
As of now, only the above mentioned libraries are supported. Although Sherlock takes custom client objects so that you can easily provide settings that you want to use for that backend store, but Sherlock also checks if the provided client object is an instance of the supported clients and accepts client objects which pass this check, even if the APIs are the same. Sherlock might get rid of this issue later, if need be and if there is a demand for that.
Installation is simple.
pip install sherlock
Sherlock will install all the client libraries for all the supported backends.
Sherlock is simple to use as at the API and semantics level, it tries to conform to standard library’s threading.Lock APIs.
import sherlock from sherlock import Lock # Configure Sherlock's locks to use Redis as the backend, # never expire locks and retry acquiring an acquired lock after an # interval of 0.1 second. sherlock.configure(backend=sherlock.backends.REDIS, expire=None, retry_interval=0.1) # Note: configuring sherlock to use a backend does not limit you # another backend at the same time. You can import backend specific locks # like RedisLock, MCLock and EtcdLock and use them just the same way you # use a generic lock (see below). In fact, the generic Lock provided by # sherlock is just a proxy that uses these specific locks under the hood. # acquire a lock called my_lock lock = Lock('my_lock') # acquire a blocking lock lock.acquire() # check if the lock has been acquired or not lock.locked() == True # release the lock lock.release()
Support for with statement
# using with statement with Lock('my_lock'): # do something constructive with your locked resource here pass
Blocking and Non-blocking API
# acquire non-blocking lock lock1 = Lock('my_lock') lock2 = Lock('my_lock') # successfully acquire lock1 lock1.acquire() # try to acquire lock in a non-blocking way lock2.acquire(False) == True # returns False # try to acquire lock in a blocking way lock2.acquire() # blocks until lock is acquired to timeout happens
Using two backends at the same time
Configuring Sherlock to use a backend does not limit you from using another backend at the same time. You can import backend specific locks like RedisLock, MCLock and EtcdLock and use them just the same way you use a generic lock (see below). In fact, the generic Lock provided by Sherlock is just a proxy that uses these specific locks under the hood.
import sherlock from sherlock import Lock # Configure Sherlock's locks to use Redis as the backend sherlock.configure(backend=sherlock.backends.REDIS) # Acquire a lock called my_lock, this lock uses Redis lock = Lock('my_lock') # Now acquire locks in Memcached from sherlock import MCLock mclock = MCLock('my_mc_lock') mclock.acquire()
To run all the tests (including integration), you have to make sure that all the databases are running. Make sure all the services are running:
# memcached memcached # redis-server redis-server # etcd (etcd is probably not available as package, here is the simplest way # to run it). wget https://github.com/coreos/etcd/releases/download/<version>/etcd-<version>-<platform>.tar.gz tar -zxvf etcd-<version>-<platform>.gz ./etcd-<version>-<platform>/etcd
Run tests like so:
python setup.py test
In short: This is an open-source project and exists for anyone to use it and/or modify it for personal use. Just be nice and attribute the credits wherever you can. :)
You are encouraged to ask questions using issues as that helps everyone and myself when people with better know-how contribute to the discussion. However, if you wish to get in touch with me personally, then I can be contacted at email@example.com.
Distributed Locking in Other Languages
- [BUGFIX] Python 3 support for sherlock
- [BUGFIX] sherlock.Lock should use globally configured client object.