log archiver for pypi cdn logs
Project description
# PyPI CDN Log Archiver
Get raw logs from S3 like object store, written by the cdn
Archive and compress them
Upload archive back to S3 like object store, in a sane structure
Remove raw logs that were archived
# Usage
You’ll need to export some env variables to get started:
$ export ACCESS_KEY=<your_access_key> $ export SECRET_KEY=<your_access_key> $ export PYPI_LOG_BUCKET=<bucket_name> $ export S3_HOST=<s3_like_host> (e.g. s3.amazonaws.com, objects.dreamhost.com)
Optionally you can set the date to archive like so:
$ export PYPI_LOG_DATE=2014-02-11
You can enable some debug output with:
$ export DEBUG=10
If you don’t set PYPI_LOG_DATE, the script will default to looking for yesterdays date.
Now you should be able to run it:
$ python pypi-cdn-log-archiver.py 112373 bytes written to testpypi-cdn-logs/archive/2014/2
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pypi-cdn-log-archiver-0.1.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 333db00b22617df0d0c9bef9b3c20f76542d5d7afa47992a4c2fc3d54d16b726 |
|
MD5 | 5443e38316e6a408b2f46a3466a16dff |
|
BLAKE2b-256 | edc9f6cb66ff07b94a25357455b175f018a6172681353ab63c7369f7523dfbb0 |
Hashes for pypi_cdn_log_archiver-0.1.3-py2.7.egg
Algorithm | Hash digest | |
---|---|---|
SHA256 | c42cc9bfc62915c9c868340e4f666f4db69db30ddbc26d7033ffb863a213fb82 |
|
MD5 | 6311a81d3d04fd2811fd158f790d01fd |
|
BLAKE2b-256 | a79b5b3b6d3a9653868d5a8611bdc4ade0427ec55602681903f3074bf7be776d |