common services for structured logging and other modules to interact with AWS resources
Project description
AWS Lambda logging
This projects provides a function to log AWS Lambda executions to cloudwatch.
Requirements
- python
Usage
from common_services import create_log_message
def handle_request(event,context):
log = create_log_message.CreateAuditMessage(context=context)
log.createauditmessage(business_key="test",business_value="test",log_level="INFO",trace_id="12334",information="this is a info message",message="received")
Structuring AWS Lambda Logs
-
Use the CreateAuditMessage_class.py to structure your logs as a json
-
Below are the logs structured when used the CreateAuditMessage_Class.
Received:
{ "timestamp": "2020-03-05T22:40:51", "logEvent": "Received", "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity", "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f", "logLevel": "INFO", "componentName": "ship-gr-idoc-to-fusion-connectivity", "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId", "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKey": { "InboundDelivery": "533197700-533197700", "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139", "PlantCode": "1062" }, "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1" }
Delivered:
{ "timestamp": "2020-03-05T22:40:51", "logEvent": "Delivered", "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity", "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f", "logLevel": "INFO", "componentName": "ship-gr-idoc-to-fusion-connectivity", "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId", "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKey": { "InboundDelivery": "533197700-533197700", "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139", "PlantCode": "1062" }, "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1" }
Discarded:
{ "timestamp": "2020-03-05T22:40:51", "logEvent": "Discarded", "message" : "Invalid Carton number", "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity", "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f", "logLevel": "INFO", "componentName": "ship-gr-idoc-to-fusion-connectivity", "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId", "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKey": { "InboundDelivery": "533197700-533197700", "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139", "PlantCode": "1062" }, "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1" }
Failure:
{ "timestamp": "2020-03-05T22:40:53", "message": "An exception of type Exception occurred. Arguments:('Tansient Error while posting to fusion. Message will be retried OSB Service Callout action received SOAP Fault response',)", "logEvent": "Failure", "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity", "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f", "logLevel": "ERROR", "componentName": "ship-gr-idoc-to-fusion-connectivity", "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId", "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7", "businessKey": { "InboundDelivery": "533197700-533197700", "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139", "PlantCode": "1062", "TrackingId": "a158f731-c424-44af-84fb-aca07fb9a0f7" }, "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1" }
-
It is recommended to write a Received log when the lambda is invoked and Delivered audit when the lambda execution is completed. This helps us to track the transaction status in the lambda. The business keys are unique attributes that can identify an event. Chose the business keys as per your business needs. When need be, we can create reports and stats using these business keys
-
When the logs are structured as outlined above, we can create a Splunk dashboard using the below method
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for common_services-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6de121f7cb775e38e1a6b50579b8423c087ddb66a6a82955c24eb0d8aa0dbcde |
|
MD5 | 1c1af241f461614ac88513eaf41a7f8e |
|
BLAKE2b-256 | ec83d469ae863fbfd747c80e21c018008e7b3cb436d83e0d54259d7b15549372 |