Archived
This project has been archived by its maintainers, and is no longer receiving any updates.
kinsumer
========
High level [Amazon Kinesis Streams](https://aws.amazon.com/kinesis/streams/) consumer.
Some features
-------------
* Automatically detect shard count changes
* Checkpoints/sequences persistence can be customized
* Provided Checkpointer implementation for memory, and file
* Memory bucket for temporary saving records
Usage
-----
```python
from kinsumer import Consumer
STREAM_REGION = 'ap-south-1'
STREAM_NAME = 'my-stream'
consumer = Consumer(__name__)
consumer.config.from_object(__name__)
@consumer.transform
def transform(data, shard_id, last_sequence_number, last_arrival_timestamp):
"""do transform and return"""
return data
@consumer.after_consume
def after(data, shard_id, last_sequence_number, last_arrival_timestamp):
"""after transform and do something"""
if __name__ == '__main__':
consumer.process()
```
Author and license
------------------
kinsumer is written by [Ungi Kim](https://ungikim.me/), maintained by [Balance Hero](http://truebalance.io/), and licensed under the MIT license. You can find the source code from [Github](https://github.com/balancehero/kinsumer):
```
$ git clone git@github.com:balancehero/kinsumer.git
```
Missing features
----------------
* Redis Checkpointer
* Consumer Heartbeat
(Contributions would be appreciated!)
========
High level [Amazon Kinesis Streams](https://aws.amazon.com/kinesis/streams/) consumer.
Some features
-------------
* Automatically detect shard count changes
* Checkpoints/sequences persistence can be customized
* Provided Checkpointer implementation for memory, and file
* Memory bucket for temporary saving records
Usage
-----
```python
from kinsumer import Consumer
STREAM_REGION = 'ap-south-1'
STREAM_NAME = 'my-stream'
consumer = Consumer(__name__)
consumer.config.from_object(__name__)
@consumer.transform
def transform(data, shard_id, last_sequence_number, last_arrival_timestamp):
"""do transform and return"""
return data
@consumer.after_consume
def after(data, shard_id, last_sequence_number, last_arrival_timestamp):
"""after transform and do something"""
if __name__ == '__main__':
consumer.process()
```
Author and license
------------------
kinsumer is written by [Ungi Kim](https://ungikim.me/), maintained by [Balance Hero](http://truebalance.io/), and licensed under the MIT license. You can find the source code from [Github](https://github.com/balancehero/kinsumer):
```
$ git clone git@github.com:balancehero/kinsumer.git
```
Missing features
----------------
* Redis Checkpointer
* Consumer Heartbeat
(Contributions would be appreciated!)
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kinsumer-0.5.3.tar.gz
(13.1 kB
view details)
File details
Details for the file kinsumer-0.5.3.tar.gz.
File metadata
- Download URL: kinsumer-0.5.3.tar.gz
- Upload date:
- Size: 13.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a330bb47da69a769d58eb2aab2def90a7f7657dd5bd9840363fc6ed25c304d7f
|
|
| MD5 |
4b1186506317482f651d6a95cf9e9bff
|
|
| BLAKE2b-256 |
40dcc67dab02aa0270742c8a8665ce82aa6bd55cf1ec981e5aef3f3dcf5e954e
|