aws-kinesis-consumer 1.0.0

Simply look up the records from a AWS Kinesis Data Stream

aws-kinesis-consumer

aws-kinesis-consumer offers the ability to simply look up the records from a AWS Kinesis Data Stream.

Motivation

The AWS CLI is able to fetch records from Kinesis, but the users need to list the shards, to generate iterator tokens, use subsequent tokens, delay operations, and so on.

aws-kinesis-consumers in contrary is able to get records from the stream name only. So users can simply scrap a stream to watch some records without any extra script.

Install

Python 3.6+ needs to be already installed, then :

pip install aws-kinesis-consumer

Usage

Just as the AWS CLI, aws-kinesis-consumer will use the AWS credentials pre-configured on the machine. Here are more details to setup AWS credentials.

Consume a stream

aws-kinesis-consumer --stream-name MyStream

Define position of the expected messages in the stream

# Consume only the new messages
aws-kinesis-consumer --stream-name MyStream --iterator-type latest

# Consume the messages already produced in the stream 
aws-kinesis-consumer --stream-name MyStream --iterator-type trim-horizon

Consume a stream hosted in a different region

AWS_DEFAULT_REGION=eu-central-1 aws-kinesis-consumer --stream-name MyGermanStream

Define endpoint different from the default AWS service endpoint

# Consuming from a different AWS endpoint url
aws-kinesis-consumer --stream-name MyStream --endpoint https://another.kinesis.us-east-1.amazonaws.com/


# Testing a local Kinesis Stream server
aws-kinesis-consumer --stream-name MyStream --endpoint http://localhost:4567/

Display help

aws-kinesis-consumer --help

Special thanks

• Thanks to the contributors of the kinesalite project which make test and development of this project extremely easy and reliable!