Skip to main content

common services for structured logging and other modules to interact with AWS resources

Project description

AWS Lambda logging

This projects provides a function to log AWS Lambda executions to cloudwatch.

Requirements

  • python

Usage

from common_services import create_log_message

def handle_request(event,context):
    log = create_log_message.CreateAuditMessage(context=context)
    log.createauditmessage(business_key="test",business_value="test",log_level="INFO",trace_id="12334",information="this is a info message",message="received")

Structuring AWS Lambda Logs

  1. Use the CreateAuditMessage_class.py to structure your logs as a json

  2. Below are the logs structured when used the CreateAuditMessage_Class.

    Received:

    {
      "timestamp": "2020-03-05T22:40:51",
      "logEvent": "Received",
      "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity",
      "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f",
      "logLevel": "INFO",
      "componentName": "ship-gr-idoc-to-fusion-connectivity",
      "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7",
      "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId",
      "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7",
      "businessKey": {
          "InboundDelivery": "533197700-533197700",
          "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139",
          "PlantCode": "1062"
       },
      "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1"
    }
    

    Delivered:

    {
          "timestamp": "2020-03-05T22:40:51",
          "logEvent": "Delivered",
          "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity",
          "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f",
          "logLevel": "INFO",
          "componentName": "ship-gr-idoc-to-fusion-connectivity",
          "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7",
          "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId",
          "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7",
          "businessKey": {
              "InboundDelivery": "533197700-533197700",
              "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139",
              "PlantCode": "1062"
           },
          "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1"
      }
    

    Discarded:

    {
          "timestamp": "2020-03-05T22:40:51",
          "logEvent": "Discarded",
          "message" : "Invalid Carton number",
          "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity",
          "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f",
          "logLevel": "INFO",
          "componentName": "ship-gr-idoc-to-fusion-connectivity",
          "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7",
          "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId",
          "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7",
          "businessKey": {
              "InboundDelivery": "533197700-533197700",
              "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139",
              "PlantCode": "1062"
           },
          "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1"
      }
    

    Failure:

    {
      "timestamp": "2020-03-05T22:40:53",
      "message": "An exception of type Exception occurred. Arguments:('Tansient Error while posting to fusion. Message will be retried OSB Service Callout action received SOAP Fault response',)",
      "logEvent": "Failure",
      "logGroup": "/aws/lambda/ship-gr-idoc-to-fusion-connectivity",
      "logStream": "2020/03/05/[$LATEST]33d7348572e14ec48efa0ea7f9745d9f",
      "logLevel": "ERROR",
      "componentName": "ship-gr-idoc-to-fusion-connectivity",
      "traceId": "a158f731-c424-44af-84fb-aca07fb9a0f7",
      "businessKeyStructure": "InboundDelivery|RefId|PlantCode|TrackingId",
      "businessKeyValue": "533197700-533197700|Vamsi-64ee-49f3-abc8-d2c49edc1139|1062|a158f731-c424-44af-84fb-aca07fb9a0f7",
      "businessKey": {
          "InboundDelivery": "533197700-533197700",
          "RefId": "Vamsi-64ee-49f3-abc8-d2c49edc1139",
          "PlantCode": "1062",
          "TrackingId": "a158f731-c424-44af-84fb-aca07fb9a0f7"
      },
      "awsRequestId": "22793623-886c-530e-b173-60f80a49aac1"
    }
    
  3. It is recommended to write a Received log when the lambda is invoked and Delivered audit when the lambda execution is completed. This helps us to track the transaction status in the lambda. The business keys are unique attributes that can identify an event. Chose the business keys as per your business needs. When need be, we can create reports and stats using these business keys

  4. When the logs are structured as outlined above, we can create a Splunk dashboard using the below method

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

common-services-1.0.0.tar.gz (4.0 kB view details)

Uploaded Source

Built Distribution

common_services-1.0.0-py3-none-any.whl (4.4 kB view details)

Uploaded Python 3

File details

Details for the file common-services-1.0.0.tar.gz.

File metadata

  • Download URL: common-services-1.0.0.tar.gz
  • Upload date:
  • Size: 4.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.2

File hashes

Hashes for common-services-1.0.0.tar.gz
Algorithm Hash digest
SHA256 0e7d7cddd727e4dc2dbef474ffa26dac6de0926853442ff9dfbb458d7198a721
MD5 619111588ffdbe3b8b264b1fd3b64b6c
BLAKE2b-256 7650cdd96b204b6181a01b1a8da01fbf3b50d268e2a81f27eb895ed4fa72bc76

See more details on using hashes here.

File details

Details for the file common_services-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: common_services-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 4.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.2

File hashes

Hashes for common_services-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 6de121f7cb775e38e1a6b50579b8423c087ddb66a6a82955c24eb0d8aa0dbcde
MD5 1c1af241f461614ac88513eaf41a7f8e
BLAKE2b-256 ec83d469ae863fbfd747c80e21c018008e7b3cb436d83e0d54259d7b15549372

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page