Skip to main content

Asyncio client for Zookeeper.

Project description

Asyncio zookeeper client (aiozk)

PyPi version

Build Status

Table of Contents

Status

Have no major bugs in client/session/connection, but recipes need more test code to become more robust.

Any help and interest are welcome 😀

Installation

$ pip install aiozk

Quick Example

import asyncio
from aiozk import ZKClient


async def main():
    zk = ZKClient('localhost')
    await zk.start()
    await zk.create('/foo', data=b'bazz', ephemeral=True)
    assert b'bazz' == await zk.get_data('/foo')
    await zk.close()

asyncio.run(main())

Recipes

You may use recipes, similar to zoonado, kazoo, and other libs:

# assuming zk is aiozk.ZKClient

# Lock
async with await zk.recipes.Lock('/path/to/lock').acquire():
    # ... Do some stuff ...
    pass

# Barrier
barrier = zk.recipes.Barrier('/path/to/barrier)
await barrier.create()
await barrier.lift()
await barrier.wait()

# DoubleBarrier
double_barrier = zk.recipes.DoubleBarrier('/path/to/double/barrier', min_participants=4)
await double_barrier.enter(timeout=0.5)
# ...  Do some stuff ...
await double_barrier.leave(timeout=0.5)

You can find full list of recipes provided by aiozk here: aiozk recipes

To understand ideas behind recipes please read this and even more recipes here. Make sure you're familiar with all recipes before doing something new by yourself, especially when it involves more than few zookeeper calls.

Caution

Don't mix different type of recipes at the same znode path. For example, creating a Lock and a DoubleBarrier object at the same path. It may cause undefined behavior 😓

Testing

NB: please ensure that you're using recent docker-compose version. You can get it by running

pip install --user -U docker-compose

Run tests

# you should have access to docker

docker-compose build
./test-runner.sh

Or you can run tests with tox

pip install --user tox tox-docker
tox

Testing approach

Most of tests are integration tests and running on real zookeeper instances. We've chosen zookeeper 3.5 version since it has an ability to dynamic reconfiguration and we're going to do all connecting/reconnecting/watches tests on zk docker cluster as this gives us the ability to restart any server and see what happens.

# first terminal: launch zookeeper cluster
docker-compose rm -fv && docker-compose build zk && docker-compose up --scale zk=7 zk_seed zk

# it will launch cluster in this terminal and remain. last lines should be like this:

zk_6       | Servers: 'server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\nserver.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\nserver.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\nserver.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\nserver.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\nserver.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181'
zk_6       | CONFIG: server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181
zk_6       | server.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181
zk_6       | server.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181
zk_6       | server.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181
zk_6       | server.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181
zk_6       | server.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181
zk_6       | server.7=172.23.0.6:2888:3888:observer;0.0.0.0:2181
zk_6       |
zk_6       |
zk_6       | Reconfiguring...
zk_6       | ethernal loop
zk_7       | Servers: 'server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181\nserver.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181\nserver.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181\nserver.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181\nserver.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181\nserver.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181\nserver.7=172.23.0.6:2888:3888:participant;0.0.0.0:2181'
zk_7       | CONFIG: server.1=172.23.0.9:2888:3888:participant;0.0.0.0:2181
zk_7       | server.2=172.23.0.2:2888:3888:participant;0.0.0.0:2181
zk_7       | server.3=172.23.0.3:2888:3888:participant;0.0.0.0:2181
zk_7       | server.4=172.23.0.4:2888:3888:participant;0.0.0.0:2181
zk_7       | server.5=172.23.0.5:2888:3888:participant;0.0.0.0:2181
zk_7       | server.6=172.23.0.7:2888:3888:participant;0.0.0.0:2181
zk_7       | server.7=172.23.0.6:2888:3888:participant;0.0.0.0:2181
zk_7       | server.8=172.23.0.8:2888:3888:observer;0.0.0.0:2181
zk_7       |
zk_7       |
zk_7       | Reconfiguring...
zk_7       | ethernal loop

Run tests in docker:

docker-compose run --no-deps aiozk
# last lines will be about testing results

............lot of lines ommited........
.
----------------------------------------------------------------------
Ran 3 tests in 1.059s

OK

Run tests locally:

# ZK_IP can be something from logs above, like: ZK_HOST=172.21.0.6:2181
ZK_HOST=<ZK_IP> ./venv/bin/pytest

Recipes testing

It seems that usually recipes require several things to be tested:

  • That recipe flow is working as expected
  • Timeouts: reproduce every timeout with meaningful values (timeout 0.5s and block for 0.6s)

Run some tests directly

Another way to run tests only which you are interested in quickly. Or this is useful when you run tests under the other version of python.

# Run zookeeper container
docker run -p 2181:2181 zookeeper

# Run pytest directly at the development source tree
export ZK_HOST=localhost
pytest -s --log-cli-level=DEBUG aiozk/test/test_barrier.py

References

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aiozk-0.31.1.tar.gz (44.8 kB view details)

Uploaded Source

Built Distribution

aiozk-0.31.1-py3-none-any.whl (61.2 kB view details)

Uploaded Python 3

File details

Details for the file aiozk-0.31.1.tar.gz.

File metadata

  • Download URL: aiozk-0.31.1.tar.gz
  • Upload date:
  • Size: 44.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.27.0

File hashes

Hashes for aiozk-0.31.1.tar.gz
Algorithm Hash digest
SHA256 379674718c0613742d65a7c4308dcf76662f81277638cb2f40225f3c80ef5177
MD5 3aadcc3d37980fdf4d40fe4d3775f6d3
BLAKE2b-256 6f76ec09bfbde0f74fde5e48f2f9f9bdce44131ebb067914f9424822a9410c09

See more details on using hashes here.

File details

Details for the file aiozk-0.31.1-py3-none-any.whl.

File metadata

  • Download URL: aiozk-0.31.1-py3-none-any.whl
  • Upload date:
  • Size: 61.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.27.0

File hashes

Hashes for aiozk-0.31.1-py3-none-any.whl
Algorithm Hash digest
SHA256 00afbc6706de132e1ef42765a780e3e15616548d9f36e93e579cc31afb3f6647
MD5 724b0d014630412b8c3d7b5b917a88de
BLAKE2b-256 84cf4d2b43b2c25eed284515d4f2ab6f46d2a43dd6e40e252eb7bf028db046e5

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page