Skip to main content
Python Software Foundation 20th Year Anniversary Fundraiser  Donate today!

Cloud Custodian - Cloud Watch Log S3 exporter

Project description

c7n-log-exporter: Cloud watch log exporter automation

A small serverless app to archive cloud logs across accounts to an archive bucket. It utilizes cloud log export to s3 feature for historical exports.

It also supports kinesis streams / firehose to move to realtime exports in the same format as the periodic historical exports.


  • Log group filtering by regex
  • Incremental support based on previously synced dates
  • Incremental support based on last log group write time
  • Cross account via sts role assume
  • Lambda and CLI support.
  • Day based log segmentation (output keys look like $prefix/$account_id/$group/$year/$month/$day/$export_task_uuid/$stream/$log)


Cli usage

make install

You can run on a single account / log group via the export subcommand

c7n-log-exporter export --help

Config format

To ease usage when running across multiple accounts, a config file can be specified, as an example.

  bucket: custodian-log-archive
  prefix: logs2

  - name: custodian-demo
    role: "arn:aws:iam::111111111111:role/CloudCustodianRole"
      - "/aws/lambda/*"
      - "vpc-flow-logs"

Multiple accounts via cli

To run on the cli across multiple accounts, edit the config.yml to specify multiple accounts and log groups.

c7n-log-exporter run --config config.yml

Serverless Usage

Edit config.yml to specify the accounts, archive bucket, and log groups you want to use.

make install
make deploy

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for c7n-logexporter, version 0.4.10
Filename, size File type Python version Upload date Hashes
Filename, size c7n_logexporter-0.4.10-py3-none-any.whl (13.8 kB) File type Wheel Python version py3 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page