Skip to main content
Version Build Status Docs

txaioetcd is an object-relational remote persistant map layer backed by etcd.

It also provides a low-level asynchronous API for general Twisted etcd applications, bypassing the object-relational layer of the library.

Installation

The implementation is pure Python code compatible with both Python 2 and 3 and runs perfect on PyPy. On the server-side, etcd version 3.1 or higher is required. To install txaioetcd

pip install txaioetcd

Getting Started

Get the complete example source code for the getting started below from here.

To start with txaioetcd using the high-level, remote persistent map API, define at least one class for data to be persisted, eg a User class:

class User(object):

    def marshal(self):
        """
        Marshal the object into a generic host language object.
        """

    @staticmethod
    def unmarshal(obj):
        """
        Parse a generic host language object into a native object of this class.
        """

Then define a table for a slot to be used with key-value stores:

from txaioetcd import pmap

# users table schema (a table with UUID keys and CBOR values holding User objects)
tab_users = pmap.MapUuidCbor(1, marshal=lambda user: user.marshal(), unmarshal=User.unmarshal)

Above will define a table slot (with index 1) that has UUIDs for keys, and CBOR serialized objects of User class for values.

The available types for keys and values of persistant maps include:

  • String (UTF8), eg MapUuidString, MapStringString, MapStringUuid, ..

  • Binary, eg MapUuidBinary, MapStringBinary, ..

  • OID (uint64), eg MapUuidOid, MapOidCbor, ..

  • UUID (uint128), eg MapUuidCbor, MapUuidUuid, ..

  • JSON, eg MapUuidJson, MapOidJson, MapStringJson, ..

  • CBOR, eg MapOidCbor, MapUuidCbor, MapStringCbor, ..

  • Pickle (Python), eg MapStringPickle, ..

  • Flatbuffers, eg MapUuidFlatbuffers, ..

For example, the following is another valid slot definition:

# users table schema (a table with OID keys and Python Pickle values holding User objects)
tab_users = pmap.MapOidPickle(2, marshal=lambda user: user.marshal(), unmarshal=User.parse)

Above will define a table slot (with index 2) that has OIDs for keys, and Python Pickle serialized objects of User class for values.

Connecting

First open a connection to etcd as a backing store:

from txaioetcd import Client, Database

etcd = Client(reactor, url='http://localhost:2379')
db = Database(etcd)

To check the database connection:

revision = await db.status()
print('connected to etcd: revision', revision)

Storing and loading objects

Now create a native Python object from the class above and store it in the table, that is remotely in etcd:

user = User()
user.name = 'foobar'
user.oid = uuid.uuid4()

# create an async writable transaction to modify etcd data
async with db.begin(write=True) as txn:
    tab_users[txn, user.oid] = user

# data is committed when transaction leaves scope .. here
print('user stored: {}'.format(user))

Load a native Python object from the table, that is remotely from etcd:

# create an async read-only transaction when only accessing data in etcd
async with db.begin() as txn:
    user = await tab_users[txn, user.oid]
    print('user loaded: {}'.format(user))

Putting it together

To put all the pieces together and run the code, you might use the following boilerplate

import txaio
txaio.use_twisted()

from twisted.internet.task import react
from twisted.internet.defer import ensureDeferred

from txaioetcd import Client, Database

async def main(reactor):
    etcd = Client(reactor, url='http://localhost:2379')
    db = Database()
    revision = await db.status()
    print('connected to etcd: revision', revision)

    # INSERT YOUR CODE HERE

def _main():
    return react(
        lambda reactor: ensureDeferred(
            main(reactor)
        )
    )

if __name__ == '__main__':
    txaio.start_logging(level='info')
    _main()

Insert your code to operate on etcd in above placeholder.

Metadata

Release files for txaioetcd 18.10.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for txaioetcd 18.10.2
File Size Uploaded
txaioetcd-18.10.2.tar.gz 39.3 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for txaioetcd 18.10.2
File Interpreter ABI Platform
txaioetcd-18.10.2-py2.py3-none-any.whl Python 3, Python 2 none any Details

Total release size: 87.2 kB

Release files / txaioetcd-18.10.2.tar.gz

Download URL txaioetcd-18.10.2.tar.gz
Size 39.3 kB
Tags Source
SHA-256 checksum
How to use checksums
1d16ba6e7ce55913baab5f763e1148ca83e7eb55aafacff8ef1ec6a2e9b83555
BLAKE2b-256 checksum
How to use checksums
93a6319da1a5fddb29aad7e43b75ffe3ae5bb5c40136a5cfbab694e3f5831756
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/39.0.1 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.7.0

Release files / txaioetcd-18.10.2-py2.py3-none-any.whl

Download URL txaioetcd-18.10.2-py2.py3-none-any.whl
Size 48.0 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
ce87048205183590fc8af7681bcd312b89290f5120066feb1ec9bb8027b631d0
BLAKE2b-256 checksum
How to use checksums
b3bf842345fc2ff02e83028214511ff9deb38b6e731c12f7325777525eb795b5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/39.0.1 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.7.0

Release history Release notifications | RSS feed

This release

18.10.2 This release

2 release files

0.4.1

2 release files

0.4.0

2 release files

0.3.0

2 release files

0.2.4

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page