s3streamcat 0.1.6

Compression Format Supported

• Gzip

• XZ

• BZIP

You might find it most useful when you don’t actually want to download the file however want to check the sample data, or to grep for few records matching certain search. Typical usage often looks like this:

s3streamcat s3://bucketname/dir/file_path
s3streamcat s3://bucketname/dir/file_path | more
s3streamcat s3://bucketname/dir/file_path | grep something

To try different encoding in case utf-8 is unable to decode your stream:

s3streamcat -e latin-1 s3://bucketname/dir/file_path.gz

To tune number of bytes to be fetched at a time:

s3streamcat -b 1024 s3://bucketname/dir/file_path.gz

Configurations

If you have aws client installed on your system s3streamcat will work out of the box If not you will have to have file at location HOME/.aws/credentials with aws cred:

[default]
aws_access_key_id=<put your aws access key>
aws_secret_access_key=<put your aws secret key>

Dependancies

• Its written for Python3 which is the present and future of the language

• libssl-dev libffi-dev

• python3-dev

Ubuntu users can install dependancy packages with:

• sudo apt-get install -y libssl-dev libffi-dev

• sudo apt-get install python3-dev