Skip to main content

Network lock based on TCP sockets

Project description

TcpNetLock

https://img.shields.io/pypi/v/tcpnetlock.svg https://img.shields.io/travis/hgdeoro/tcpnetlock.svg Documentation Status Updates

Network lock based on TCP sockets

Why?

While deploying applications to Kubernetes, I needed a way to make sure that some potential concurrent, distributed actions, are not executed concurrently. For example:

  • database migrations: just one Pod in the Kubernetes cluster should be able to apply the database migrations

  • for batch jobs, different workers could be working on the same resource, this can be avoided with this lock mechanism

Of course, Zookeeper is a MUCH BETTER solution, but that’s too much for my use cases…

How it works

Assuming the server is running on localhost, let’s get a lock using telnet:

$ telnet localhost 7654
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

To try to acquire a lock, send:

lock,name:django-migrations

Server responds with:

ok

From that point, and while the TCP connection is open, you have the lock.

If you try the same in a different terminal, you will get:

$ telnet localhost 7654
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
lock,name:django-migrations        <= you write
not-granted                        <= server response
Connection closed by foreign host. <= server closed the connection

Here the server responded with not-granted and closed the TCP connection. The lock was not granted to you.

But, in real-life scenarios, you would use the provided utility tcpnetlock_do:

$ tcpnetlock_do --lock-name django-migrations -- python manage.py migrate

To test it, you will need the server running. To get the server running with Docker, just run:

$ docker pull hgdeoro/tcpnetlock
$ docker run -ti --rm -p 7654:7654 hgdeoro/tcpnetlock

Alternatively, you can install the package in a virtualenv and launch the server:

$ virtualenv -p python3.6 venv
$ source venv/bin/activate
$ pip install tcpnetlock
$ tcpnetlock_server --info
INFO:root:Started server listening on localhost:7654

Features

  • Runs on Python 3.6 / Python 3.5

  • Do not require external libraries

  • Ready to use Docker image (based on Alpine)

  • Includes server and python client

  • Includes utility to run Linux commands while holding the lock

  • Simple protocol: you can get a lock even with netcat

Appendix: netcat

Since the protocol is just text over a TCP connection, you can get a lock just writing the right text overt the TCP connection and leaving that TCP connection open, and that’s the default behaviour of netcat:

$ echo 'lock,name:LOCK_NAME' | nc localhost 7654

The first line uses netcat to open the TCP connection and tries to get the lock.

The biggest problem would be to READ the response to the server (will be one of ‘ok’ or ‘not-granted’) while send nc to the background. We can use a fifo for that:

$ echo 'lock,name:LOCK_NAME' | nc -v localhost 7654 | tee /tmp/.tcpnetlock &
$ result=$(head -n 1 /tmp/.tcpnetlock)

Even though this works, using one of the two existing python clients (tnl_client and tnl_do) would be much better.

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.1.8 (2018-07-20)

  • Make period of cleanup configurable via environment variables

  • FIX response for stats action

  • Add tests for background thread

  • Add client method to call the stats action

0.1.7 (2018-07-19)

  • Log release of lock causde by client disconnecting

  • Add .stats action

  • Add background thread to cleanup old locks

  • Remove global server state hold on Context class

0.1.6 (2018-07-06)

  • FIX tag used in Docker image

  • log server version when starting

0.1.5 (2018-06-26)

  • FIX variable name used for building Docker image

  • Change theme for docs

0.1.4 (2018-06-26)

  • Implements retries for cli tnl_do

0.1.3 (2018-06-22)

  • Server logs granted lock

  • Add description to CLI

  • Client use environment variables for host/port

  • Add __str__ to Action to have better logs in server

0.1.2 (2018-06-21)

  • Update client & server to handle errors in a better way

  • Add tests

  • Update docs

0.1.1 (2018-06-20)

  • Add .bashrc (for developers)

  • Fix setup.py

0.1.0 (2018-06-19)

  • Docker start server with –info by default

  • Adds cloudbuild.yaml to facilitate building in GCP

  • Change in protocol to detect unintended uses

  • Detect invalid requests and always send response to client

  • BIG refactor of server and client classes

  • Add lot of tests (current coverage: 99%)

0.0.8 (2018-06-18)

  • Refactor messy code from server, client and cli

0.0.7 (2018-06-17)

  • Code cleanup and refactor

  • Add tests

  • Implements run_with_lock script to make really easy to use from shell scripts

0.0.6 (2018-06-16)

  • Create shell script to be sourced, to facilitate use of tcpnetlock from shell scripts

0.0.5 (2018-06-15)

  • Update CONTRIBUTING (documents commands for the full release process)

  • Disable upload to pypi from Travis-CI

0.0.4 (2018-06-15)

  • Encapsulate Lock, adds client id and timestamp

  • Implement sending of keepalive from client

  • Remove use of ‘click’

  • Start server from cli with configurable parameters (listen address, port, etc)

  • Use client id to identify who has the lock

0.0.3 (2018-06-15)

  • Validate lock name in server

  • FIX client to handle RESPONSE_ERR response

  • Add unittests

  • Refactor locks into server class

  • Use threading for test server

  • Make code compatible with Python 3.5

0.0.2 (2018-06-15)

  • Implements RELEASE of locks

  • FIX release of lock when client closes the connection

  • Validates lock name

  • Code refactoring

0.0.1 (2018-06-15)

  • Add files from cookiecutter-pypackage

  • Migrate test cases to pytest

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tcpnetlock-0.1.8.tar.gz (36.2 kB view details)

Uploaded Source

Built Distribution

tcpnetlock-0.1.8-py2.py3-none-any.whl (21.5 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file tcpnetlock-0.1.8.tar.gz.

File metadata

  • Download URL: tcpnetlock-0.1.8.tar.gz
  • Upload date:
  • Size: 36.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for tcpnetlock-0.1.8.tar.gz
Algorithm Hash digest
SHA256 7a7e83eba4eb43fe5ef5690e2f3251ecf4712be1d04d05df999401c2c8243d2a
MD5 35a76bdbb8a850130b30a84a6274c118
BLAKE2b-256 a2bb3b33453a10179b1c48087e5a026684f4e19cfb9ae515015301b3d57d15ec

See more details on using hashes here.

File details

Details for the file tcpnetlock-0.1.8-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for tcpnetlock-0.1.8-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 7e90f73498cb180448fe9ca50dd28728b9e32bf06cbba17ccd0a52e0a507e605
MD5 caae5f0943af1543dd1daa18f45329bc
BLAKE2b-256 c5efba877a2445efc763ac9a9a670a0d9d5062011d9ea9b17309305f095d15de

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page