Implementation of raft consensus algorithm.
Project description
consensual
Summary
consensual
is a pure-Python library for defining network of nodes
running in a consistent fault-tolerant manner by implementing state-of-the-art
Raft
consensus algorithm.
Currently, next features are implemented & property-based tested
- leader election & log replication (described in section 5 of the article),
- cluster membership changes (described in section 6 of the article), namely consensual addition & removal of nodes,
- "solo mode": non-consensual separation of a node resulting in running as a cluster-by-itself,
- separate node state resetting with history deletion.
Next crucial features to implement will be
- persistence,
- log compaction (described in section 7 of the article).
In what follows python
is an alias for python3.7
or any later version (python3.8
and so on).
Installation
Install the latest pip
& setuptools
packages versions
python -m pip install --upgrade pip setuptools
User
Download and install the latest stable version from PyPI
repository
python -m pip install --upgrade consensual
Developer
Download the latest version from GitHub
repository
git clone https://github.com/lycantropos/consensual.git
cd consensual
Install dependencies
python -m pip install -r requirements.txt
Install
python setup.py install
Usage
>>> from consensual.raft import Node, communication
>>> from yarl import URL
>>> node_url = URL.build(scheme='http',
... host='localhost',
... port=6000)
>>> other_node_url = URL.build(scheme='http',
... host='localhost',
... port=6001)
>>> heartbeat = 0.1
>>> from typing import Any, List, Optional
>>> processed_parameters = []
>>> def dummy_processor(parameters: Any) -> None:
... processed_parameters.append(parameters)
>>> processors = {'dummy': dummy_processor}
>>> nodes = {}
>>> sender = communication.Sender([node_url], nodes)
>>> other_sender = communication.Sender([other_node_url], nodes)
>>> node = Node.from_url(node_url,
... heartbeat=heartbeat,
... processors=processors,
... sender=sender)
>>> other_node = Node.from_url(other_node_url,
... heartbeat=heartbeat,
... processors=processors,
... sender=other_sender)
>>> receiver = communication.Receiver(node, nodes)
>>> other_receiver = communication.Receiver(other_node, nodes)
>>> receiver.start()
>>> other_receiver.start()
>>> from asyncio import get_event_loop
>>> loop = get_event_loop()
>>> async def run() -> List[Optional[str]]:
... return [await node.solo(),
... await node.enqueue('dummy', 42),
... await node.attach_nodes([other_node.url]),
... await node.enqueue('dummy', 42),
... await other_node.detach_nodes([node.url]),
... await other_node.solo(),
... await other_node.detach(),
... await other_node.detach()]
>>> error_messages = loop.run_until_complete(run())
>>> receiver.stop()
>>> other_receiver.stop()
>>> all(error_message is None or isinstance(error_message, str)
... for error_message in error_messages)
True
>>> all(parameters == 42 for parameters in processed_parameters)
True
We can also replace builtin consensual.raft.communication
communication layer
with another one (like consensual_http
which is built on top of HTTP), usage patterns may change as a result.
Development
Bumping version
Preparation
Install bump2version.
Pre-release
Choose which version number category to bump following semver specification.
Test bumping version
bump2version --dry-run --verbose $CATEGORY
where $CATEGORY
is the target version number category name, possible
values are patch
/minor
/major
.
Bump version
bump2version --verbose $CATEGORY
This will set version to major.minor.patch-alpha
.
Release
Test bumping version
bump2version --dry-run --verbose release
Bump version
bump2version --verbose release
This will set version to major.minor.patch
.
Running tests
Install dependencies
python -m pip install -r requirements-tests.txt
Plain
pytest
Inside Docker
container:
docker-compose --file docker-compose.yml up
Bash
script:
./run-tests.sh
PowerShell
script:
.\run-tests.ps1
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for consensual-0.2.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15a222cafbe3447d930e3a4916fad0f494a1f9beea0853afb8bbe6d712aeb31d |
|
MD5 | 602212912853b2750893bb51643dd14e |
|
BLAKE2b-256 | 81c1b4353478a734c7dcd273964d72e298f878ab0fa84dc91853f27947e44e23 |