Skip to main content

High level Amazon Kinesis Streams consumer

This project has been archived.

The maintainers of this project have marked this project as archived. No new releases are expected.

Project description

kinsumer
========

High level [Amazon Kinesis Streams](https://aws.amazon.com/kinesis/streams/) consumer.

Some features
-------------

* Automatically detect shard count changes
* Checkpoints/sequences persistence can be customized
* Provided Checkpointer implementation for memory, and file
* Memory bucket for temporary saving records

Usage
-----

```python
from kinsumer import Consumer

STREAM_REGION = 'ap-south-1'
STREAM_NAME = 'my-stream'
consumer = Consumer(__name__)
consumer.config.from_object(__name__)

@consumer.transform
def transform(data, shard_id, last_sequence_number, last_arrival_timestamp):
"""do transform and return"""
return data

@consumer.after_consume
def after(data, shard_id, last_sequence_number, last_arrival_timestamp):
"""after transform and do something"""

if __name__ == '__main__':
consumer.process()
```

Author and license
------------------

kinsumer is written by [Ungi Kim](https://ungikim.me/), maintained by [Balance Hero](http://truebalance.io/), and licensed under the MIT license. You can find the source code from [Github](https://github.com/balancehero/kinsumer):

```
$ git clone git@github.com:balancehero/kinsumer.git
```

Missing features
----------------

* Redis Checkpointer
* Consumer Heartbeat

(Contributions would be appreciated!)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kinsumer-0.5.3.tar.gz (13.1 kB view details)

Uploaded Source

File details

Details for the file kinsumer-0.5.3.tar.gz.

File metadata

  • Download URL: kinsumer-0.5.3.tar.gz
  • Upload date:
  • Size: 13.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for kinsumer-0.5.3.tar.gz
Algorithm Hash digest
SHA256 a330bb47da69a769d58eb2aab2def90a7f7657dd5bd9840363fc6ed25c304d7f
MD5 4b1186506317482f651d6a95cf9e9bff
BLAKE2b-256 40dcc67dab02aa0270742c8a8665ce82aa6bd55cf1ec981e5aef3f3dcf5e954e

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page