Python utilities for AWS Lambda functions including but not limited to tracing, logging and custom metric
Project description
Lambda Powertools
A suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier - Currently available for Python only and compatible with Python >=3.6.
Status: Beta
Features
Tracing
It currently uses AWS X-Ray
- Decorators that capture cold start as annotation, and response and exceptions as metadata
- Run functions locally without code change to disable tracing
- Explicitly disable tracing via env var
POWERTOOLS_TRACE_DISABLED="true"
Logging
- Decorators that capture key fields from Lambda context, cold start and structures logging output as JSON
- Optionally log Lambda request when instructed (disabled by default)
- Enable via
POWERTOOLS_LOGGER_LOG_EVENT="true"
or explicitly via decorator param
- Enable via
- Logs canonical custom metric line to logs that can be consumed asynchronously
- Log sampling enables DEBUG log level for a percentage of requests (disabled by default)
- Enable via
POWERTOOLS_LOGGER_SAMPLE_RATE=0.1
, ranges from 0 to 1, where 0.1 is 10% and 1 is 100%
- Enable via
Environment variables used across suite of utilities
Environment variable | Description | Default | Utility |
---|---|---|---|
POWERTOOLS_SERVICE_NAME | Sets service name used for tracing namespace, metrics dimensions and structured logging | "service_undefined" | all |
POWERTOOLS_TRACE_DISABLED | Disables tracing | "false" | tracing |
POWERTOOLS_LOGGER_LOG_EVENT | Logs incoming event | "false" | logging |
POWERTOOLS_LOGGER_SAMPLE_RATE | Debug log sampling | 0 | logging |
LOG_LEVEL | Sets logging level | "INFO" | logging |
Usage
Installation
With pip installed, run: pip install aws-lambda-powertools
Tracing
Example SAM template using supported environment variables
Globals:
Function:
Tracing: Active # can also be enabled per function
Environment:
Variables:
POWERTOOLS_SERVICE_NAME: "payment"
POWERTOOLS_TRACE_DISABLED: "false"
Pseudo Python Lambda code
from aws_lambda_powertools.tracing import Tracer
tracer = Tracer()
# tracer = Tracer(service="payment") # can also be explicitly defined
@tracer.capture_method
def collect_payment(charge_id):
# logic
ret = requests.post(PAYMENT_ENDPOINT)
# custom annotation
tracer.put_annotation("PAYMENT_STATUS", "SUCCESS")
return ret
@tracer.capture_lambda_handler
def handler(event, context)
charge_id = event.get('charge_id')
payment = collect_payment(charge_id)
...
Logging
Example SAM template using supported environment variables
Globals:
Function:
Environment:
Variables:
POWERTOOLS_SERVICE_NAME: "payment"
POWERTOOLS_LOGGER_SAMPLE_RATE: 0.1 # enable debug logging for 1% of requests, 0% by default
LOG_LEVEL: "INFO"
Pseudo Python Lambda code
from aws_lambda_powertools.logging import logger_setup, logger_inject_lambda_context
logger = logger_setup()
# logger_setup(service="payment") # also accept explicit service name
# logger_setup(level="INFO") # also accept explicit log level
@logger_inject_lambda_context
def handler(event, context)
logger.info("Collecting payment")
...
# You can log entire objects too
logger.info({
"operation": "collect_payment",
"charge_id": event['charge_id']
})
...
Exerpt output in CloudWatch Logs
{
"timestamp":"2019-08-22 18:17:33,774",
"level":"INFO",
"location":"collect.handler:1",
"service":"payment",
"lambda_function_name":"test",
"lambda_function_memory_size":"128",
"lambda_function_arn":"arn:aws:lambda:eu-west-1:12345678910:function:test",
"lambda_request_id":"52fdfc07-2182-154f-163f-5f0f9a621d72",
"cold_start": "true",
"message": "Collecting payment"
}
{
"timestamp":"2019-08-22 18:17:33,774",
"level":"INFO",
"location":"collect.handler:15",
"service":"payment",
"lambda_function_name":"test",
"lambda_function_memory_size":"128",
"lambda_function_arn":"arn:aws:lambda:eu-west-1:12345678910:function:test",
"lambda_request_id":"52fdfc07-2182-154f-163f-5f0f9a621d72",
"cold_start": "true",
"message":{
"operation":"collect_payment",
"charge_id": "ch_AZFlk2345C0"
}
}
Custom Metrics async
NOTE: This will likely change after Beta in light of new Amazon CloudWatch embedded metric format, meaning we won't need an additional stack and interface could change.
This feature requires Custom Metrics SAR App in order to process canonical metric lines in CloudWatch Logs.
If you're starting from scratch, you may want to see a working example, tune to your needs and deploy within your account - Serverless Airline Log Processing Stack
from aws_lambda_powertools.logging import MetricUnit, log_metric
def handler(event, context)
log_metric(name="SuccessfulPayment", unit=MetricUnit.Count, value=10, namespace="MyApplication")
# Optional dimensions
log_metric(name="SuccessfulPayment", unit=MetricUnit.Count, value=10, namespace="MyApplication", customer_id="123-abc", charge_id="abc-123")
# Explicit service name
log_metric(service="paymentTest", name="SuccessfulPayment", namespace="MyApplication".....)
...
Exerpt output in CloudWatch Logs
MONITORING|10|Count|SuccessfulPayment|MyApplication|service="payment
MONITORING|10|Count|SuccessfulPayment|MyApplication|customer_id="123-abc",charge_id="abc-123",service="payment
MONITORING|10|Count|SuccessfulPayment|MyApplication|service="paymentTest
Beta
This library may change its API/methods or environment variables as it receives feedback from customers. Currently looking for ideas in the following areas before making it stable:
- Should Tracer patch all possible imported libraries by default or only AWS SDKs?
- Patching all libraries may have a small performance penalty (~50ms) at cold start
- Alternatively, we could patch only AWS SDK if available and to provide a param to patch multiple
Tracer(modules=("boto3", "requests"))
- Create a Tracer provider to support additional tracing
- Either duck typing or ABC to allow additional tracing providers
TODO
- Add an example code using powertools
Credits
- Structured logging initial implementation from aws-lambda-logging
- Powertools idea DAZN Powertools
License
This library is licensed under the MIT-0 License. See the LICENSE file.
HISTORY
February 20th, 2020
0.4.0
- Introduces log sampling for debug - Thanks to Danilo's contribution
November 15th, 2019
- Public beta release
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for aws_lambda_powertools-0.4.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 04324fc7f6fc6791401df299a237b65148d96d6a6f3de3949f881f89573f278b |
|
MD5 | 953bbbccaa1b4709564a78b0a84ddd4e |
|
BLAKE2b-256 | e68c69bacc6d92e1cb5bf7a33fef511420720e9b91a28c060e32a232f66895c7 |
Hashes for aws_lambda_powertools-0.4.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c05c64f25246b742854a7ab62fcd78a7c437a1ca706965f90ee34992cdec0233 |
|
MD5 | 43e2cf85bcb0a72863d30cd1ea1d7c3c |
|
BLAKE2b-256 | e29faffa31a4b9e0b90a9a5f13b5c8d39a43875bbe088043a46c10371c395756 |