raftify-py

⚠️ WARNING: This library is in a very experimental stage. The API could be broken.

Python binding of raftify.

Quick guide

I strongly recommend to read the basic memstore example code to get how to use this library for starters, but here's a quick guide.

Define your own log entry

Define the data to be stored in LogEntry and how to serialize and de-serialize it.

class SetCommand:
    def __init__(self, key: str, value: str) -> None:
        self.key = key
        self.value = value

    def encode(self) -> bytes:
        return pickle.dumps(self.__dict__)

    @classmethod
    def decode(cls, packed: bytes) -> "SetCommand":
        unpacked = pickle.loads(packed)
        return cls(unpacked["key"], unpacked["value"])

Define your application Raft FSM

Essentially, the following three methods need to be implemented for the Store.

apply: applies a committed entry to the store.
snapshot: returns snapshot data for the store.
restore: applies the snapshot passed as argument.

And also similarly to LogEntry, you need to implement encode and decode.

class HashStore:
    def __init__(self):
        self._store = dict()

    def get(self, key: str) -> Optional[str]:
        return self._store.get(key)

    def apply(self, msg: bytes) -> bytes:
        message = SetCommand.decode(msg)
        self._store[message.key] = message.value
        logging.info(f'SetCommand inserted: ({message.key}, "{message.value}")')
        return msg

    def snapshot(self) -> bytes:
        return pickle.dumps(self._store)

    def restore(self, snapshot: bytes) -> None:
        self._store = pickle.loads(snapshot)

Bootstrap a raft cluster

First, bootstrap the cluster that contains the leader node.

logger = Slogger.default()
logger.info("Bootstrap new Raft Cluster")

node_id = 1
raft_addr = "127.0.0.1:60061"
raft = Raft.bootstrap(node_id, raft_addr, store, cfg, logger)
await raft.run()

Join follower nodes to the cluster

Then join the follower nodes.

If peer specifies the configuration of the initial members, the cluster will operate after all member nodes are bootstrapped.

raft_addr = "127.0.0.1:60062"
peer_addr = "127.0.0.1:60061"

join_ticket = await Raft.request_id(raft_addr, peer_addr)
node_id = join_ticket.get_reserved_id()

raft = Raft.bootstrap(node_id, raft_addr, store, cfg, logger)
tasks = []
tasks.append(raft.run())
await raft.join([join_ticket])

Manipulate FSM by RaftServiceClient

If you want to operate the FSM remotely, use the RaftServiceClient.

client = await RaftServiceClient.build("127.0.0.1:60061")
await client.propose(SetCommand("1", "A").encode())

Manipulate FSM by RaftNode

If you want to operate FSM locally, use the RaftNode interface of the Raft object

raft_node = raft.get_raft_node()
await raft_node.propose(message.encode())

Debugging

Raftify also provides a collection of CLI commands that let you check the data persisted in stable storage and the status of Raft Server.

$ raftify_cli debug persisted ./logs/node-1

$ raftify_cli debug node 127.0.0.1:60061

Metadata

Release files for raftify 0.1.67

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for raftify 0.1.67
File	Size	Uploaded
raftify-0.1.67.tar.gz	48.5 kB	Aug 4, 2024	Details

Built distribution (wheel)

Table of built distributions (wheels) for raftify 0.1.67
File	Interpreter	ABI	Platform	Reset
raftify-0.1.67-cp312-cp312-macosx_11_0_arm64.whl 3.2 MB Aug 4, 2024	CPython 3.12	CPython 3.12	macOS 11.0+ ARM64	Details

Total release size: 3.2 MB

Release files / raftify-0.1.67.tar.gz

Download URL	raftify-0.1.67.tar.gz
Size	48.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`6be8df3f3b53b1189b19e597781dceb5285a30a8d54d3441fef8dcc7a1fd68ea`
BLAKE2b-256 checksum How to use checksums	`9af5c8e3c5a050229fbf33787b62bc6736e6f7b8f503d516a7588daf7375483c`
Upload date	Aug 4, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`maturin/1.7.0`

Release files / raftify-0.1.67-cp312-cp312-macosx_11_0_arm64.whl

Download URL	raftify-0.1.67-cp312-cp312-macosx_11_0_arm64.whl
Size	3.2 MB
Tags	CPython 3.12 macOS 11.0+ ARM64
SHA-256 checksum How to use checksums	`583592ab9650bc3041a8532d0e803e6588a19a8032ec9f590469cd01990c3bd6`
BLAKE2b-256 checksum How to use checksums	`7200a8d51149e180df3c0b816137aee4da4a7d07b095fb880ea868364b6b8e5c`
Upload date	Aug 4, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`maturin/1.7.0`