esio-kinesis 1.2.0

Async AWS Kinesis client with consumer decorator, checkpointing, and factory pattern for EdgeStack IoT platform

esio-kinesis

Async AWS Kinesis client for the EdgeStack IoT platform. Provides publish, consume, and stream management with a factory pattern that supports sequential, concurrent, and Enhanced Fan-Out (EFO) consumer strategies.

Features

• Async-native — built on aioboto3, safe in asyncio task groups
• Consumer decorator pattern — ack() / rollback(cause) for explicit at-least-once processing
• Redis checkpointing — resumes from last processed sequence number on restart
• LocalStack compatible — AT_TIMESTAMP falls back to TRIM_HORIZON automatically
• Factory pattern — swap KINESIS_CLIENT_TYPE=simple|concurrent|efo via env var, zero service code changes

Installation

pip install esio-kinesis

Quick start

from esio_kinesis import KinesisClientFactory

consumer = KinesisClientFactory.get(
    client_type='simple',
    endpoint_url='http://localhost:4566',   # omit for real AWS
    region='us-east-1',
    checkpoint_store=redis_client,          # optional Redis for checkpointing
    consumer_id='my-service',
)

# Publish
await consumer.put_record('my-stream', '{"hello": "world"}', partition_key='key')

# Consume — ack/rollback are explicit
async for message in consumer.consume('my-stream'):
    await process(message)
    await consumer.ack()

Consumer types

KINESIS_CLIENT_TYPE	Description	Status
simple	Sequential polling with Redis checkpointing	Available
concurrent	Semaphore-bounded concurrent batch processing	Planned
efo	Enhanced Fan-Out via SubscribeToShard	Planned

License

MIT