Skip to main content

Kines

PyPI version PyPI downloads CI

Friendly Command Line Interface for Amazon Kinesis Data Streams

Kines Demo

Install

  • pip install kines

Setup

  • aws configure

Commands

List all Kinesis Stream
  • kines ls

This command lists streams with their open shard count, enhanced consumers count, retention period and encryption type. Internally this command calls list-streams and describe_stream_summary methods of Boto3 Kinesis client.

Kines ls

List Kinesis Stream Shards
  • kines lss <stream-name>

This command will show you open as well as closed shards of a Kinesis stream.

  • For detailed output Run kines lss <stream-name> -d

Kines lss

Find shard for partition key
  • kines find prod-clickstream -p 123455 -p 8900

From Kinesis Docs

A partition key is used to group data by shard within a stream. Kinesis Data Streams segregates the data records belonging to a stream into multiple shards. It uses the partition key that is associated with each data record to determine which shard a given data record belongs to. Partition keys are Unicode strings with a maximum length limit of 256 bytes. An MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. When an application puts data into a stream, it must specify a partition key.

This command comes handy when you want to determine shard for a partition key. This Command can accept multiple partition keys.

Kines find

Put records into Kinesis Stream

  • Put a single record: kines put <stream-name> -p <partition-key> -d '{"event": 1}' (If -p is omitted, a random UUID is used as the partition key).

  • Put KPL-aggregated records: kines put <stream-name> -a -p pk1 -d '{"event": 1}' -p pk2 -d '{"event": 2}'

  • Put from file: kines put <stream-name> -f payload.json (Accepts a raw payload file or a JSON array of [{"partition_key": "...", "data": "..."}, ...] with -a to aggregate).

Walk through kinesis records

  • kines walk <stream-name> <shard-id> -s <sequence-number> -n <number-of-records-per-call>

You can use this command to debug kinesis records. This command internally creates a Shard Iterator. If you don't specify sequence-number then a ShardIterator is created with type TRIM_HORIZON, which allows you to fetch from oldest Kinesis records in shard. When you specify sequence-number a ShardIterator is created with AT_SEQUENCE_NUMBER type, which fetches data from the specified sequence number. The Kinesis record's data is decoded using base64 decoder. You can press ⏎ to fetch more records or type n to abort.

Kines Walk

  • Use -t option to get records from 1 hour 10 minutes ago. Example: kines walk click-stream 000000000000 -t '1h10m'

  • Use -l option to start from latest records. Example: kines walk click-stream 000000000000 -l

  • Use -f option to poll records repeatedly. Example: kines walk click-stream 000000000000 -l -f

Get report for Kinesis Stream

The report command gives you statistics about your Kinesis Stream. Internally, this command calls get_metric_data of Boto3 Cloudwatch client.

Please take a look at legends command for shortforms used in this report.

Kines Report 60 hours 6 hours

View all short forms and legends

  • kines legends

Kines Legends

How to build in Dev?

  • pip install --editable .

How to publish to pypi?

  • python3 setup.py sdist bdist_wheel
  • twine upload --skip-existing dist/*

Metadata

Release files for kines 1.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for kines 1.2.0
File Size Uploaded Provenance Actions
kines-1.2.0.tar.gz 20.9 kB Without provenance Details for kines-1.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for kines 1.2.0
File Interpreter ABI Platform Actions
kines-1.2.0-py3-none-any.whl Python 3 none any Details for kines-1.2.0-py3-none-any.whl

Total release size: 36.1 kB

Release files / kines-1.2.0.tar.gz

Download URL kines-1.2.0.tar.gz
Size 20.9 kB
Tags Source
SHA-256 checksum
How to use checksums
4645f86e14a37c2ee15acf09358bb8a974aa6f194600c5b90b443014e5d3197f
BLAKE2b-256 checksum
How to use checksums
f7b66f30e4857851a13cf887cfdbd379b62bc54c9e2b7efb8857ba9bbbb10eea
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.15.2

Release files / kines-1.2.0-py3-none-any.whl

Download URL kines-1.2.0-py3-none-any.whl
Size 15.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
018f5a20053fc1320aa5875bfd23002cb0b52670145b056f25e5f54afac3af9e
BLAKE2b-256 checksum
How to use checksums
7cfe089809d07728ef441e4a046825be164c605f21a62a24a1b14cdfb065697a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.15.2

Security for kines 1.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page