Streaming based unix cat like functionality for file on S3. Supports gzip, bzip and xz compressed files as well
S3 Stream Cat powers you to instantly check the content of S3 file. Especially it supports streaming and printing data from compressed files.
Compression Format Supported
You might find it most useful when you don’t actually want to download the file however want to check the sample data, or to grep for few records matching certain search. Typical usage often looks like this:
s3streamcat s3://bucketname/dir/file_path s3streamcat s3://bucketname/dir/file_path | more s3streamcat s3://bucketname/dir/file_path | grep something
To try different encoding in case utf-8 is unable to decode your stream:
s3streamcat -e latin-1 s3://bucketname/dir/file_path.gz
To tune number of bytes to be fetched at a time:
s3streamcat -b 1024 s3://bucketname/dir/file_path.gz
If you have aws client installed on your system s3streamcat will work out of the box If not you will have to have file at location HOME/.aws/credentials with aws cred:
[default] aws_access_key_id=<put your aws access key> aws_secret_access_key=<put your aws secret key>
Its written for Python3 which is the present and future of the language
Ubuntu users can install dependancy packages with:
sudo apt-get install -y libssl-dev libffi-dev
sudo apt-get install python3-dev
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.