Skip to main content

A command-line tool to get valuable information out of AWS CloudTrail

Project description

TrailScraper

PyPi Release Build Status

A command-line tool to get valuable information out of AWS CloudTrail

Installation

$ pip install trailscraper

Usage

# Download some logs (including us-east-1 for global aws services)
$ trailscraper download --bucket some-bucket \
                        --account-id some-account-id \
                        --region some-other-region \
                        --region us-east-1 \
                        --from 'two days ago' \
                        --to 'now' \
# Generate an IAM Policy
$ trailscraper generate
{
    "Statement": [
        {
            "Action": [
                "ec2:DescribeInstances",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVolumes",
                "ec2:DescribeVpcs",
            ],
            "Effect": "Allow",
            "Resource": [
                "*"
            ]
        },
        {
            "Action": [
                "sts:AssumeRole"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:iam::1111111111:role/someRole"
            ]
        }
    ],
    "Version": "2012-10-17"
}

Development

$ ./go setup   # set up venv, dependencies and tools
$ ./go test    # run some tests
$ ./go check   # run some style checks
$ ./go         # let's see what we can do here

Troubleshooting

TrailScraper is missing some events
  • Make sure you have logs for the us-east-1 region. Some global AWS services (e.g. Route53, IAM, STS, CloudFront) use this region. For details, check the CloudTrail Documentation
Click thinks you are in an ASCII environment

Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment.

Set environment variables that describe your locale, e.g. :

export LC_ALL=de_DE.utf-8
export LANG=de_DE.utf-8

or

LC_ALL=C.UTF-8
LANG=C.UTF-8

For details, see http://click.pocoo.org/5/python3/#python-3-surrogate-handling

Changelog

This changelog contains a loose collection of changes in every release including breaking changes to the API.

The format is based on Keep a Changelog

0.4.1

Fixed

  • Ignore record files that can’t be read (e.g. not valid GZIP) in Python 2.7 (was only working in Python 3.* before)
  • Fixed permissions generated for services that include the API version date (e.g. Lambda, CloudFront) (#20)

0.4.0

Added

  • Support for CloudTrail lookup_events API that allows users to generate a policy without downloading logs from an S3 bucket. Note that this API only returns `“create, modify, and delete API calls” <https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-supported-services.html>`__
  • trailscraper download now supports --from and --to flags to specify the timeframe that should be downloaded. Accepts precise (e.g. “2017-10-12”) and relative (e.g. “-2days”) arguments.
  • trailscraper generate-policy now supports --from and --to to filter events to consider for the generated policy. Accepts precise (e.g. “2017-10-12”) and relative (e.g. “-2days”) arguments.
  • Performance optimizations: generate-policy only reads logfiles for the timeframe requested
  • Added --version command line argument

Changed

  • Set more flexible dependencies

Removed

  • Removed --past-days parameter in trailscraper download. Was replaced by --from and --to (see above)

Fixed

  • Ignore record files that can’t be read (e.g. not valid GZIP)

0.3.0

Added

  • Support for Python >= 2.7

Changed

  • Do not download CloudTrail Logs from S3 if they already exist in the target folder (#9)
  • Removed dependency on fork of the awacs-library to simplify installation and development

Fixed

  • Bug that led to policy-statements with the same set of actions not being combined properly in some cases (#7)

0.2.0

Added

  • Basic filtering for role-arns when generating policy (#3)

0.1.0

Initial Release

Added

  • Basic feature to download CloudTrail Logs from S3 for certain accounts and timeframe
  • Basic feature to generate IAM Policies from a set of downloaded CloudTrail logs

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for trailscraper, version 0.4.1
Filename, size File type Python version Upload date Hashes
Filename, size trailscraper-0.4.1-py2.py3-none-any.whl (14.3 kB) File type Wheel Python version 2.7 Upload date Hashes View
Filename, size trailscraper-0.4.1.tar.gz (11.7 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page