Skip to main content

A library to parse S3 log files.

Project description

A python library to parse S3 log files.


Unit tests currently require actual S3 credentials (and a bucket with logs) and can therefore only be run manually. Mocking the relevant parts of boto.s3 is on the roadmap. Contributions are welcome :)


Download S3 logs from a bucket, and parse them.

This application does not store the log objects generated and leaves that to other applications.


Eventually from pypi …


To get logs, simply use the Downloader class:

from itertools import chain
from pprint import pprint
from import Downloader


downloader = Downloader(

# download 10 logs
logs = downloader.download_files(

entries = [ log.entries for log in logs ]
entries = list(chain.from_iterable(entries))
[pprint(entry.__dict__) for entry in entries]

Running tests

First you need to specify the test config, which contains the AWS credentials and details of bucket tot test with. python-testconfig is used to manage the test configuration.

Copy test_config.ini.sample to test_config.ini (in the same directory) and set correct values:

export NOSE_TESTCONFIG_AUTOLOAD_INI=`pwd`/test_config.ini

To test with nose:

python nosetests

or running nosetests directly:

nosetests -s --exe

Directly and with coverage:

nosetests -s --exe --with-coverage --cover-package=lss3logs

(Note: the –exe includes python files whoch are executable, so it’s optional if you don’t have any.)

Code quality

Checking code with pylint:

pylint lss3logs



The regular expression for parsing the log lines is copied from a script by “kkowalczyk” located at

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ls-s3-logs, version 0.1.2
Filename, size File type Python version Upload date Hashes
Filename, size ls-s3-logs-0.1.2.tar.gz (12.5 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page