Skip to main content

# Dumplogs

Dumplogs is a command line tool (python) for dumping the aws cloudwatch log streams into local files.

## Motivation

On AWS cloud, Users can get historical AWS cloudwatch logs by exporting the log group into S3 bucket. See [AWS logs S3 Export](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3Export.html) The exported file are all in gz format and are placed in separate directory. one way to download them into loacl machine is to use “aws s3 cp s3://WholeBucket LocalFolder –recursive” , and then unzip all the gz files. This operation on AWS is inevitable fussiness.

A more direct approach to dump the logs into local files is to use “aws logs” cli command. But aws-cli logs only allows max. 50 log streams to be retrieved at a time. In order to dump the whole logs streams from a log group, user need record the ‘next_token’ every time by himself.

Dumplogs is inspired by “aws logs” , it aims to dump (and filter) the whole log streams from a AWS log group to local text files.

## Installation

`bash pip3 install https://github.com/xinhuagu/dumplogs.git `

or

`bash python3 -m pip install https://github.com/xinhuagu/dumplogs.git `

## Usage

Make sure the AWS_PROFILE is set properly

`bash export AWS_PROFILE=my_profile `

`bash usage: dumplogs [-h] [--profile AWS_PROFILE] [-o OUTPUT] group_name `

  • -h –help help information

  • -o –output output directory, default: current directory

  • –profile aws profile, default: AWS_PROFILE env var

  • group_name aws log group name

## Example

`bash dumplogs /aws/lambda/dpa-kafka-logs -o test --profile DEV `

or

`bash dumplogs /aws/lambda/dpa-kafka-logs `

## Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

## License

[Revised BSD License](https://github.com/xinhuagu/dumplogs/blob/master/LICENSE.txt)

Metadata

Release files for awsdumplogs 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for awsdumplogs 0.1.1
File Interpreter ABI Platform Actions
awsdumplogs-0.1.1-py3.9.egg Legacy Egg format - - Details for awsdumplogs-0.1.1-py3.9.egg

Release files / awsdumplogs-0.1.1-py3.9.egg

Download URL awsdumplogs-0.1.1-py3.9.egg
Size 5.9 kB
Tags Egg
SHA-256 checksum
How to use checksums
58953146423882d6e0820b82088d9e84eb8c30acce9c31bbd0230a9d8d943459
BLAKE2b-256 checksum
How to use checksums
e2e43de3f3a4288e8b6df42e77c7850a18a71334c33875ff0465813c0c04f355
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.5

Security for awsdumplogs 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

1 release file No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page