Skip to main content

distributed increment/decrement counter leveraging dynamodb

Project description

Build Status Documentation Status

Overview

distributed_counter is a python package for maintaining counters across clusters of computers. It leverages AWS DynamoDB for storage and atomicity of the counter.

Installation

distributed_counter is compatible with python2 and python3. Simply use pip to install.

pip install distributed_counter

You'll also want to set up your aws configurations.

Usage

The interface for distributed_counter is very simple. Everything is done through the DistributedCounter class. To instantiate:

from distributed_counter import DistributedCounter

counter = DistributedCounter('my_dynamo_table_name')

You can pass anything in kwargs to DistributedCounter and they will be propagated to the boto3 Session, e.g.

counter = DistributedCounter('mytable', region_name='us-west-1', aws_access_key_id='somekey',
                             aws_secret_access_key='somesecret')

There are also special parameters config and endpoint_url which get passed to the DynamoDB ServiceResource.

Table Creation

The dynamodb table has one HASH key called key. You can create the table yourself, or you can use create_table.

counter.create_table()

distributed_counter is smart enough to wait for the table to finish creation on your next call to the table.

Getting/Setting

Next, you can set a key using put and get it with get:

counter.put('mykey', 0)
counter.get('mykey')
0

Increment/Decrementing

Finally, you can increment/decrement.

counter.increment('mykey', 10)
10
counter.increment('mykey', 5)
15
counter.decrement('mykey', 14)
1

Note that the returned value is the new value.

You can use increment/decrement with a default. This is the same as doing a put then an increment/decrement.

counter.increment('nonexistantkey', 0, 0)
0
counter.increment('someotherkey', 0, 10)
10
counter.increment('yetanotherkey', 1, 10)
11

Example Use

Let's say we want to run a function every 100 calls to an API. In your API, you can put:

if not counter.increment('mykey', 1) % 100:
    run_function()
    counter.decrement('mykey', 100)

This guarantees that no matter how many servers you have, every 100 calls to your API will run your function.

The modulo is used instead of == because there is a corner case where hypothetically you could increment to 200 before the decrement finishes.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

distributed_counter-0.0.3.tar.gz (4.8 kB view details)

Uploaded Source

File details

Details for the file distributed_counter-0.0.3.tar.gz.

File metadata

  • Download URL: distributed_counter-0.0.3.tar.gz
  • Upload date:
  • Size: 4.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.7.5

File hashes

Hashes for distributed_counter-0.0.3.tar.gz
Algorithm Hash digest
SHA256 6ee478bd1d4e3fcf9b59c10d7622b3067e7c690cf9cf7e1633090f141cf00dcf
MD5 adb54601d8ddce8426e7c3675c874ca3
BLAKE2b-256 70f7bdf2d62e068abd3da01e63825fc58deb2a37fc0c5e00350c8095ae9fe5f4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page