distributed increment/decrement counter leveraging dynamodb
Project description
Overview
distributed_counter
is a python package for maintaining counters across clusters of computers.
It leverages AWS DynamoDB for storage and atomicity of the counter.
Installation
distributed_counter
is compatible with python2
and python3
.
Simply use pip
to install.
pip install distributed_counter
You'll also want to set up your aws configurations.
Usage
The interface for distributed_counter
is very simple.
Everything is done through the DistributedCounter
class.
To instantiate:
from distributed_counter import DistributedCounter
counter = DistributedCounter('my_dynamo_table_name')
You can pass anything in kwargs
to DistributedCounter and they will be propagated to the boto3 Session, e.g.
counter = DistributedCounter('mytable', region_name='us-west-1', aws_access_key_id='somekey',
aws_secret_access_key='somesecret')
There are also special parameters config
and endpoint_url
which get passed to the
DynamoDB ServiceResource.
Table Creation
The dynamodb table has one HASH key called key
. You can create the table yourself, or you can use create_table
.
counter.create_table()
distributed_counter
is smart enough to wait for the table to finish creation on your next call to the table.
Getting/Setting
Next, you can set a key using put
and get it with get
:
counter.put('mykey', 0)
counter.get('mykey')
0
Increment/Decrementing
Finally, you can increment
/decrement
.
counter.increment('mykey', 10)
10
counter.increment('mykey', 5)
15
counter.decrement('mykey', 14)
1
Note that the returned value is the new value.
You can use increment
/decrement
with a default.
This is the same as doing a put
then an increment
/decrement
.
counter.increment('nonexistantkey', 0, 0)
0
counter.increment('someotherkey', 0, 10)
10
counter.increment('yetanotherkey', 1, 10)
11
Example Use
Let's say we want to run a function every 100 calls to an API. In your API, you can put:
if not counter.increment('mykey', 1) % 100:
run_function()
counter.decrement('mykey', 100)
This guarantees that no matter how many servers you have, every 100 calls to your API will run your function.
The modulo is used instead of ==
because there is a corner case where hypothetically you could increment to 200 before
the decrement finishes.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file distributed_counter-0.0.3.tar.gz
.
File metadata
- Download URL: distributed_counter-0.0.3.tar.gz
- Upload date:
- Size: 4.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.7.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6ee478bd1d4e3fcf9b59c10d7622b3067e7c690cf9cf7e1633090f141cf00dcf |
|
MD5 | adb54601d8ddce8426e7c3675c874ca3 |
|
BLAKE2b-256 | 70f7bdf2d62e068abd3da01e63825fc58deb2a37fc0c5e00350c8095ae9fe5f4 |