Skip to main content

Get, sort and analyse AWS CloudWatch logs from multiple log groups using AWS CloudWatch Insights service

Project description

awsinsights

Get and filter logs from multiple log groups of AWS CloudWatch and filter CloudWatch logs using predefined regular expressions.

This script uses AWS CloudWatch Insights service.

Basic Usage

  1. Set up AWS Credentials.

  2. Define apps (sets of CloudWatch log groups assigned to app name) in $HOME/.awsinsights.json file. See example file below.

  3. Get all logs from the simplebook app (2 log groups pre-defined) since last 30 minutes:

awsinsights --timedelta 30m --appname simplebook
  1. Filter logs from simplebook app since last 7d containing words 'Monday' or 'Tuesday' (you can use any Regular Expression in )
awsinsights --timedelta 7d --appname simplebook --filter "Monday|Tuesday"

Advanced Usage

  1. Get logs from simplebook from 1 Jan 2021 10:00am to 2 Jan 2021 9:00am which contain 'Exception' or 'ERROR' on PROD environment
awsinsights --env prod --start 2021-01-01 10:00:00 --end 2021-01-02 09:00:00 --appname simplebook --filter "Exception|ERROR"
  1. Get all logs from CloudWatch log groups group-one-dev and /aws/lambda/group-two-dev since last 2 hours:
awsinsights --timedelta 2h --log_groups "group-one-dev" "/aws/lambda/group-two-dev"

Example of config file

Config file should be placed in $HOME/.awsinsights.json

This example file contains 2 apps: simplebook and secondapp. Each app consits of 2 CloudWatch log groups.

{
    "simplebook": [
        "/aws/lambda/simple-books-catalog-api-$ENV",
        "/aws/lambda/api-task-2-ad"
    ],
    "secondapp": [
        "first-log-group",
        "/aws/lambda/second-log-group"
    ]
}

Output file

Logs will be written to output file. Output file will be:

  • /tmp/{appname}.log if app name is defined using --appname option OR
  • /tmp/awsinsights.log if app name is NOT defined

Help

awsinsights [-h] [--timedelta TIMEDELTA] [--start START] [--end END]
                   [--filter FILTER]
                   (--appname APPNAME | --log_groups LOG_GROUPS [LOG_GROUPS ...])
                   [--env ENV] [--query QUERY]

optional arguments:
  -h, --help            show this help message and exit
  --timedelta TIMEDELTA
                        delta time since now when logs should be filtered ex.
                        120m, 3h, 2d. Default: 60m
  --start START         start time of grabbing logs. Format: YYYY-MM-DD
                        HH:MM:SS
  --end END             end time of grabbing logs. Format: YYYY-MM-DD HH:MM:SS
  --filter FILTER       Regular expression for filtering logs
  --appname APPNAME     name of the app which logs should be analysed. App
                        names should have logs groups configured in
                        .awsinsightsrc file. See README.md file.
  --log_groups LOG_GROUPS [LOG_GROUPS ...]
                        list of the log groups " "to analyse (up to 20)
  --env ENV             env name. It can be used to resolve "{env}" var in log
                        groups names. Default: dev
  --query QUERY         Custom full AWS CloudWatch Insights query. " "Default:
                        fields @timestamp, @message | filter @message like //
                        | sort @timestamp

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

awsinsights-1.0.2.tar.gz (5.4 kB view details)

Uploaded Source

Built Distribution

awsinsights-1.0.2-py3-none-any.whl (7.3 kB view details)

Uploaded Python 3

File details

Details for the file awsinsights-1.0.2.tar.gz.

File metadata

  • Download URL: awsinsights-1.0.2.tar.gz
  • Upload date:
  • Size: 5.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.7.0

File hashes

Hashes for awsinsights-1.0.2.tar.gz
Algorithm Hash digest
SHA256 024d2117f5ec8ea4e886e35a9259c4955da73b0e40e2f4e317983fc9f8d29051
MD5 87c42b9735094b65c2c766d61b3208ee
BLAKE2b-256 01baf8b2e32001f3cf7c35beff65c26b0107d8dbc56422d1da263ecc2d7f1c2d

See more details on using hashes here.

File details

Details for the file awsinsights-1.0.2-py3-none-any.whl.

File metadata

  • Download URL: awsinsights-1.0.2-py3-none-any.whl
  • Upload date:
  • Size: 7.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.7.0

File hashes

Hashes for awsinsights-1.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 09fa6ad53bde02bb4e4b8fc190007fc10c1a600e447a970adeadc2069dd3e365
MD5 9a7e4c10cb071b27d2d6dce030acc300
BLAKE2b-256 b2fee8a5ad2aa4cf38330a3a18f0add3ca27051f95476a33f26babc4709e95d5

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page