A set of Step Functions high-level patterns.
Project description
cdk-stepfunctions-patterns
cdk-stepfunctions-patterns library is a set of AWS CDK constructs that provide resiliency patterns implementation for AWS Step Functions.
All these patterns are composable, meaning that you can combine them together to create quite complex state machines that are much easier to maintain and support than low-level JSON definitions.
- Try / Catch
- Try / Finally
- Try / Catch / Finally
- Retry with backoff and jitter
- Resilience lambda errors handling
- Validation of proper resilience lambda errors handling
Try / Catch pattern
Step Functions support Try / Catch pattern natively with Task and Parallel states.
TryTask
construct adds a high level abstraction that allows you to use Try / Catch pattern with any state or sequence of states.
Example
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
from cdk_stepfunctions_patterns import TryTask
# ...
sfn.StateMachine(self, "TryCatchStepMachine",
definition=TryTask(self, "TryCatch",
try_process=sfn.Pass(self, "A1").next(sfn.Pass(self, "B1")),
catch_process=sfn.Pass(self, "catchHandler"),
# optional configuration properties
catch_props={
"errors": ["Lambda.AWSLambdaException"],
"result_path": "$.ErrorDetails"
}
)
)
Resulting StepFunction
Try / Finally pattern
It is often useful to design state machine using Try / Finally pattern. The idea is to have a Final state that has to be executed regardless of successful or failed execution of the Try state. There may be some temporal resource you want to delete or notification to send.
Step Functions do not provide a native way to implement that pattern but it can be done using Parallel state and catch all catch specification.
TryTask
construct abstracts these implementation details and allows to express the pattern directly.
Example
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
from cdk_stepfunctions_patterns import TryTask
# ...
sfn.StateMachine(self, "TryFinallyStepMachine",
definition=TryTask(self, "TryFinally",
try_process=sfn.Pass(self, "A2").next(sfn.Pass(self, "B2")),
finally_process=sfn.Pass(self, "finallyHandler"),
# optional configuration properties
finally_error_path="$.FinallyErrorDetails"
)
)
Resulting StepFunction
Try / Catch / Finally pattern
This is a combination of two previous patterns. TryTask
construct allows you to express rather complex
error handling logic in a very compact form.
Example
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
from cdk_stepfunctions_patterns import TryTask
# ...
sfn.StateMachine(self, "TryCatchFinallyStepMachine",
definition=TryTask(self, "TryCatchFinalli",
try_process=sfn.Pass(self, "A3").next(sfn.Pass(self, "B3")),
catch_process=sfn.Pass(self, "catchHandler3"),
finally_process=sfn.Pass(self, "finallyHandler3")
)
)
Resulting StepFunction
Retry with backoff and jitter
Out of the box Step Functions retry implementation provides a way to configure backoff factor, but there is no built in way to introduce jitter. As covered in Exponential Backoff And Jitter and Wait and Retry with Jittered Back-off this retry technique can be very helpful in high-load scenarios.
RetryWithJitterTask
construct provides a custom implementation of retry with backoff and
jitter that you can use directly in your state machines.
Example
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
from cdk_stepfunctions_patterns import RetryWithJitterTask
# ...
sfn.StateMachine(self, "RetryWithJitterStepMachine",
definition=RetryWithJitterTask(self, "AWithJitter",
try_process=sfn.Pass(self, "A4").next(sfn.Pass(self, "B4")),
retry_props={"errors": ["States.ALL"], "max_attempts": 3}
)
)
Resulting StepFunction
Resilience lambda errors handling
LambdaInvoke
construct from aws-stepfunctions-tasks
module is probably one of the most used ones. Still, handling of
AWS Lambda service exceptions
is often overlooked.
ResilientLambdaTask
is a drop-in replacement construct for LambdaInvoke
that adds retry for the most common
transient errors:
- Lambda.ServiceException
- Lambda.AWSLambdaException
- Lambda.SdkClientException
- Lambda.TooManyRequestsException
Example
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_lambda as lambda_
from cdk_stepfunctions_patterns import ResilientLambdaTask
# ...
lambda_function = lambda_.Function(self, "LambdaFunction")
calculate_jitter_task = ResilientLambdaTask(self, "InvokeLambda",
lambda_function=lambda_function
)
That would result in the following state definition:
"InvokeLambda": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "<ARN of lambda function>"
},
"Retry": [{
"ErrorEquals": [
"Lambda.ServiceException",
"Lambda.AWSLambdaException",
"Lambda.SdkClientException",
"Lambda.TooManyRequestsException"
],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
}]
}
Validation of proper resilience lambda errors handling
It is often a challenge to enforce consistent transient error handling across all state machines of a large application. To help with that, cdk-stepfuctions-patterns provides a CDK aspect to verify that all Lambda invocations correctly handle transient errors from AWS Lambda service.
Use ResilienceLambdaChecker
aspect as shown below.
Example
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.core as cdk
from cdk_stepfunctions_patterns import ResilienceLambdaChecker
app = cdk.App()
# ...
# validate compliance rules
app.node.apply_aspect(ResilienceLambdaChecker())
If there are some states in your application that do not retry transient errors or miss some recommended error codes, there will be warning during CDK synthesize stage:
PS C:\Dev\GitHub\cdk-stepfunctions-patterns> cdk synth --strict
[Warning at /StepFunctionsPatterns/A] No retry for AWS Lambda transient errors defined - consider using ResilientLambdaTask construct.
[Warning at /StepFunctionsPatterns/B] Missing retry for transient errors: Lambda.AWSLambdaException,Lambda.SdkClientException.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cdk-stepfunctions-patterns-0.1.5.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35fe1c6345d9b3f415134b17f6b4e50443bf9e89c2293e8c37ca2277fc995f32 |
|
MD5 | 1637c5562752f37248552cfbd2e45a3e |
|
BLAKE2b-256 | c80908c128e17826161fef1710de633f0964184c58ab7aad779cdd5a5fe533b7 |
Hashes for cdk_stepfunctions_patterns-0.1.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97631efc5072ec16ea308387e4b846693acaf6dbc358025cdb3075d0af6a17af |
|
MD5 | d359f6db3d4d4af5f8882fcbedc85714 |
|
BLAKE2b-256 | 0809fcc74ebcd96bd37065100d5b45f71504ee9fb296174117224e4fdb758a18 |