Skip to main content

An integer id sequence generator using elasticsearch to create bulks of ids. Elasticsearch can not create integer ids for newly created documents. This package helps to be able to create unique integer ids in a distributed system.

Features

  • makes sure multiple processes are able to create unique ids

  • optimized by requesting multiple ids with a single Elasticsearch request

Elasticsearch Setup

The sequences must be assigned to an elasticsearch client instance:

>>> from elasticsearch import Elasticsearch
>>> es_client = Elasticsearch(['localhost:%s' % crate_port])

To globally assign the client to all sequences in the application the client can be assigned to the class property ES:

>>> from lovely.essequence import Sequence
>>> Sequence.ES = es_client

Usage

There is a Sequence class:

>>> from lovely.essequence import Sequence

It must be instantiated with the name of a sequence:

>>> s1 = Sequence('s1')

Now the iids can be requested:

>>> s1.next()
1
>>> s1.next()
2

Multiple sequence instances for the same sequence name use the same bulk:

>>> s2 = Sequence('s1', bulk_size=100)
>>> s2.next()
3

After the bulk size has been consumed the request of the next bulk is transparently handled inside the next method:: Consume all the cached ids from s1:

>>> for i in range(10): s1.next()
4
5
6
7
8
9
10
11
12
13

Multiple independed sequences can be used:

>>> other = Sequence('other')
>>> other.next()
1

Transforming the provided iid

A transformation function can be provided to transform the provided integer value into something else when calling next:

>>> asstring = Sequence('other', transform=str)
>>> isinstance(asstring.next(), str)
True

>>> def transformer(iid):
...     return {'iid': iid}
>>> Sequence('other', transform=transformer).next()
{'iid': 3}

Resetting a Sequence

For testing purposes it is possible to reset a sequence.

Release files for lovely.essequence 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for lovely.essequence 0.1.0
File Size Uploaded
lovely.essequence-0.1.0.tar.gz 3.7 kB Details

Release files / lovely.essequence-0.1.0.tar.gz

Download URL lovely.essequence-0.1.0.tar.gz
Size 3.7 kB
Tags Source
SHA-256 checksum
How to use checksums
46ef85f3c8d03cc6374ff48d4307d85c3d72cc6be78fd7ef4efa1fb399ef247d
BLAKE2b-256 checksum
How to use checksums
e80f3f78bf140b3f8d5b2f299b7454540bd3d76b2c9dcde4884ebaffc3343ae7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release history Release notifications | RSS feed

This release

0.1.0 This release

1 release file

0.0.2

1 release file

0.0.1

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page