Logging aws_logging_handlers to AWS services that support S3 and Kinesis stream logging with multiple threads
Project description
A python multithreaded logging handler package that streams records to AWS services objects with support for the following AWS services: * S3 * Kinesis
Supports gzip compression(in S3)
Getting Started
Prerequisites
Asynchronous multipart uploading relies on the ability to use multiple threads #### Packages:
boto3
Installing
Installation using pip
pip install aws-logging-handlers
Examples
Stream log records to S3 and Kinesis
import logging from aws_logging_handlers import S3Handler, KinesisHandler bucket="test_bucket" # The bucket should already exist # The log will be rotated to a new object either when an object reaches 5 MB or when 120 seconds pass from the last rotation/initial logging s3_handler = S3Handler("test_log", bucket, workers=3) kinesis_handler = KinesisHandler('log_test', 'us-east-1', workers=1) formatter = logging.Formatter('[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s') s3_handler.setFormatter(formatter) kinesis_handler.setFormatter(formatter) logger = logging.getLogger('root') logger.setLevel(logging.INFO) logger.addHandler(s3_handler) logger.addHandler(kinesis_handler) for i in range(0, 100000): logger.info("test info message") logger.warning("test warning message") logger.error("test error message")
To be developed
Support for asyncio
Logging and upload metrics
License
This project is licensed under the MIT License - see the LICENSE.md file for details
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Close
Hashes for aws-logging-handlers-2.0.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e865371ba15095b90939e5cc26ea542f2afb45bd8044d9ef9f4fd2d22ce6a256 |
|
MD5 | 9918406893f306863d44f6a0cfd70b9c |
|
BLAKE2b-256 | 621b7e3b54466e02a6bfe64e3baa80c1b2c3b4e8e70d8fddadf9f1b673859d72 |
Close
Hashes for aws_logging_handlers-2.0.0-py3.7.egg
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0790c0d9b4ae9cbe24dc4fb33c5f13cdff8ba90197d942f00a153221f8116b63 |
|
MD5 | 8a406b04cfe7fb2b21b65196bbedeecf |
|
BLAKE2b-256 | 3727439c997dbd5c0c780e2176ad6d7adb717aed9f4c04a83085b497ae866234 |
Close
Hashes for aws_logging_handlers-2.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6006209979c50c922fdc91707380a9f34cbde51f5354fd33eaba84c2feb513a6 |
|
MD5 | 7fff3aa08d0c3971b2882c877613007e |
|
BLAKE2b-256 | 04d901d10acd08dcb44f0d52c53faf52a907a828b3184924225aba461ba913d4 |