Network lock based on TCP sockets
Project description
TcpNetLock
Network lock based on TCP sockets
Free software: GNU General Public License v3
Documentation: https://tcpnetlock.readthedocs.io/
Why?
While deploying applications to Kubernetes, I needed a way to make sure that some potential concurrent, distributed actions, are not executed concurrently. For example:
database migrations: just one Pod in the Kubernetes cluster should be able to apply the database migrations
for batch jobs, different workers could be working on the same resource, this can be avoided with this lock mechanism
Of course, Zookeeper is a MUCH BETTER solution, but that’s too much for my use cases…
How it works
Assuming the server is running on localhost, let’s get a lock using telnet:
$ telnet localhost 7654 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'.
To try to acquire a lock, send:
lock,name:django-migrations
Server responds with:
ok
From that point, and while the TCP connection is open, you have the lock.
If you try the same in a different terminal, you will get:
$ telnet localhost 7654 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. lock,name:django-migrations <= you write not-granted <= server response Connection closed by foreign host. <= server closed the connection
Here the server responded with not-granted and closed the TCP connection. The lock was not granted to you.
But, in real-life scenarios, you would use the provided utility tcpnetlock_do:
$ tcpnetlock_do --lock-name django-migrations -- python manage.py migrate
To test it, you will need the server running. To get the server running with Docker, just run:
$ docker pull hgdeoro/tcpnetlock $ docker run -ti --rm -p 7654:7654 hgdeoro/tcpnetlock
Alternatively, you can install the package in a virtualenv and launch the server:
$ virtualenv -p python3.6 venv $ source venv/bin/activate $ pip install tcpnetlock $ tcpnetlock_server --info INFO:root:Started server listening on localhost:7654
Features
Runs on Python 3.6 / Python 3.5
Do not require external libraries
Ready to use Docker image (based on Alpine)
Includes server and python client
Includes utility to run Linux commands while holding the lock
Simple protocol: you can get a lock even with netcat
Appendix: netcat
Since the protocol is just text over a TCP connection, you can get a lock just writing the right text overt the TCP connection and leaving that TCP connection open, and that’s the default behaviour of netcat:
$ echo 'lock,name:LOCK_NAME' | nc localhost 7654
The first line uses netcat to open the TCP connection and tries to get the lock.
The biggest problem would be to READ the response to the server (will be one of ‘ok’ or ‘not-granted’) while send nc to the background. We can use a fifo for that:
$ echo 'lock,name:LOCK_NAME' | nc -v localhost 7654 | tee /tmp/.tcpnetlock & $ result=$(head -n 1 /tmp/.tcpnetlock)
Even though this works, using one of the two existing python clients (tnl_client and tnl_do) would be much better.
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.1.8 (2018-07-20)
Make period of cleanup configurable via environment variables
FIX response for stats action
Add tests for background thread
Add client method to call the stats action
0.1.7 (2018-07-19)
Log release of lock causde by client disconnecting
Add .stats action
Add background thread to cleanup old locks
Remove global server state hold on Context class
0.1.6 (2018-07-06)
FIX tag used in Docker image
log server version when starting
0.1.5 (2018-06-26)
FIX variable name used for building Docker image
Change theme for docs
0.1.4 (2018-06-26)
Implements retries for cli tnl_do
0.1.3 (2018-06-22)
Server logs granted lock
Add description to CLI
Client use environment variables for host/port
Add __str__ to Action to have better logs in server
0.1.2 (2018-06-21)
Update client & server to handle errors in a better way
Add tests
Update docs
0.1.1 (2018-06-20)
Add .bashrc (for developers)
Fix setup.py
0.1.0 (2018-06-19)
Docker start server with –info by default
Adds cloudbuild.yaml to facilitate building in GCP
Change in protocol to detect unintended uses
Detect invalid requests and always send response to client
BIG refactor of server and client classes
Add lot of tests (current coverage: 99%)
0.0.8 (2018-06-18)
Refactor messy code from server, client and cli
0.0.7 (2018-06-17)
Code cleanup and refactor
Add tests
Implements run_with_lock script to make really easy to use from shell scripts
0.0.6 (2018-06-16)
Create shell script to be sourced, to facilitate use of tcpnetlock from shell scripts
0.0.5 (2018-06-15)
Update CONTRIBUTING (documents commands for the full release process)
Disable upload to pypi from Travis-CI
0.0.4 (2018-06-15)
Encapsulate Lock, adds client id and timestamp
Implement sending of keepalive from client
Remove use of ‘click’
Start server from cli with configurable parameters (listen address, port, etc)
Use client id to identify who has the lock
0.0.3 (2018-06-15)
Validate lock name in server
FIX client to handle RESPONSE_ERR response
Add unittests
Refactor locks into server class
Use threading for test server
Make code compatible with Python 3.5
0.0.2 (2018-06-15)
Implements RELEASE of locks
FIX release of lock when client closes the connection
Validates lock name
Code refactoring
0.0.1 (2018-06-15)
Add files from cookiecutter-pypackage
Migrate test cases to pytest
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tcpnetlock-0.1.8-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e90f73498cb180448fe9ca50dd28728b9e32bf06cbba17ccd0a52e0a507e605 |
|
MD5 | caae5f0943af1543dd1daa18f45329bc |
|
BLAKE2b-256 | c5efba877a2445efc763ac9a9a670a0d9d5062011d9ea9b17309305f095d15de |