SWIM protocol implementation for exchanging cluster membership status and metadata.
Project description
swim-protocol
SWIM protocol implementation for exchanging cluster membership status and metadata.
This library is intended to fit into an asyncio event loop to help synchronize a distributed group of processes.
Introduction
API Documentation
Install and Usage
$ pip install swim-protocol
Running the Demo
There is a demo application included as a reference implementation. Try it out by running the following, each from a new terminal window, and use Ctrl-C to exit:
$ swim-protocol-demo -c --name 127.0.0.1:2001 --peer 127.0.0.1:2003 --metadata name one
$ swim-protocol-demo -c --name 127.0.0.1:2002 --peer 127.0.0.1:2001 --metadata name two
$ swim-protocol-demo -c --name 127.0.0.1:2003 --peer 127.0.0.1:2001 --metadata name three
$ swim-protocol-demo -c --name 127.0.0.1:2004 --peer 127.0.0.1:2003 --metadata name four
Typing in any window will disseminate what has been typed across the cluster with eventual consistency.
Getting Started
First you should create a new UdpConfig object:
from swimprotocol.udp import UdpConfig
config = UdpConfig(local_name='127.0.0.1:2001',
local_metadata={'name': b'one'},
peers=['127.0.0.1:2002'])
All other config arguments have default values, which are tuned somewhat arbitrarily with a small cluster of 3-4 members in mind.
Now you can create the cluster members manager and transport layer, and enter the event loop:
from contextlib import AsyncExitStack
from swimprotocol.members import Members
from swimprotocol.udp import UdpTransport
transport = UdpTransport(config)
members = Members(config)
async with AsyncExitStack() as stack:
worker = await stack.enter_async_context(transport.enter(members))
await worker.run() # or schedule as a task
These snippets demonstrate the UDP transport layer directly. For a more generic approach that uses argparse and load_transport, check out the demo.
If your application is deployed as a Docker Service, the UdpConfig
discovery=True
keyword argument can be used to discover configuration based
on the service name. See the documentation for more comprehensive usage.
Checking Members
The Members object provides a few ways to check on the cluster and its members:
for member in members.non_local:
# all other known cluster members
print(member.name, member.status, member.metadata)
from swimprotocol.status import Status
for member in members.get_status(Status.AVAILABLE):
# all cluster members except offline
print(member.name, member.status, member.metadata)
Alternatively, listen for status or metadata changes on all members:
from swimprotocol.member import Member
async def _updated(member: Member) -> None:
print('updated:', member.name, member.status, member.metadata)
async with AsyncExitStack() as stack:
# ...
stack.enter_context(members.listener.on_notify(_updated))
UDP Transport Security
The UdpTransport transport layer (the only included transport implementation) uses salted hmac digests to sign each UDP packet payload. Any UDP packets received that are malformed or have an invalid signature are silently ignored. The eventual consistency model should recover from packet loss.
The signatures rely on a shared secret between all cluster members, given
as the secret=b'...'
argument to the Config constructor. If
secret=None
is used, it defaults to uuid.getnode()
but this is not
secure for production setups unless all sockets are bound to a local loopback
interface.
The cluster member metadata is not encrypted during transmission, so only private networks should be used if metadata includes any secret data, or that secret data should be encrypted separately by the application. Also be aware that low MTU sizes on public networks may affect the ability to synchronize larger amounts of metadata.
Development
You will need to do some additional setup to develop and test plugins. Install Hatch to use the CLI examples below.
Run all tests and linters:
$ hatch run check
Because this project supports several versions of Python, you can use the following to run the checks on all versions:
$ hatch run all:check
Type Hinting
This project makes heavy use of Python's type hinting system, with the intention of a clean run of mypy:
$ mypy
No code contribution will be accepted unless it makes every effort to use type hinting to the extent possible and common in the rest of the codebase.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for swim_protocol-0.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8cd9864a6f22a36b276856e909029842518ecd6cdf50e3b760fa9bbd206a2c49 |
|
MD5 | 30a3863a1d4a9a174a2e6f701c9a6666 |
|
BLAKE2b-256 | 140e375748d7ae8a150ebeb71c538d1e6c3aaad151100b0cd2b74895845c87f5 |