Skip to main content

A redrive construct to use with an SQS queue and it's dead letter queue

Project description

SQS Redrive

This construct creates a Lambda function that you can use to move SQS messages from one queue to another. This is often used for moving Dead Letter Queue messages back to the original queue for reprocessing.

This is a pre-release!

This is a quick first-draft. All the options that will likely need to be added to accomodate a large number of use-cases are still needed. If you'd like to make requests or help update this construct, please open an Issue or a PR.

What Gets Created

A Lambda function and related policy which moves SQS queue messages from one queue to another.

Example

This creates two external queues and then creates the Lambda to move from the deadLetterQueue to the mainQueue

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.core as cdk
from ...lib.sqs_redrive import SqsRedrive
from aws_cdk.aws_sqs import Queue

app = cdk.App()
stack = cdk.Stack(app, "test-stack")

main_queue = Queue(stack, "main-queue")
dead_letter_queue = Queue(stack, "dlq-queue")
SqsRedrive(stack, "SqsRedriveConstructStack",
    main_queue=main_queue,
    dead_letter_queue=dead_letter_queue
)

Note: this is the integration test (cdk synth).

Input Properties

What are the inputs to your constructs?

property description example
mainQueue The destination queue for the messages. new Queue(stack, 'main-queue')
deadLetterQueue The source queue of the messages. new Queue(stack, 'dead-letter-queue')

Overriding Lambda Props

You can supply your own properties to the Lambda Function constructor. They're mashed together with some defaults. Pay attention to the order:

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
self.redrive_function = NodejsFunction(self, f"{id}-queue-redrive",
    function_name=id,
    entry=join(__dirname, "sqs-redrive.queue-redrive.ts"),
    (SpreadAssignment ...props.lambdaProps
      props.lambda_props),
    environment={
        "QUEUE_URL": props.main_queue.queue_url,
        "DLQ_URL": props.dead_letter_queue.queue_url,
        (SpreadAssignment ...props?.lambdaProps?.environment
          props.lambda_props.environment)
    }
)

functionName and entry can be overridden. Environment variables will always be splatted with the two queue URLs so you never have to worry about specifying those (you can, of course, override them, but if you're going that far then why are you using this construct?).

Output Properties

After constructed, you can gain access to the Lambda Function:

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
redrive = SqsRedrive(stack, "SqsRedriveConstructStack",
    main_queue=main_queue,
    dead_letter_queue=dead_letter_queue
)

Design Notes

This is early design and serves one very specific use-case. If you have suggestions on how to make this better, please open an Issue in Github.

Contributing

Please open Pull Requests and Issues on the Github Repo.

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mbonig.sqs-redrive-1.85.0.tar.gz (20.9 kB view details)

Uploaded Source

Built Distribution

mbonig.sqs_redrive-1.85.0-py3-none-any.whl (21.2 kB view details)

Uploaded Python 3

File details

Details for the file mbonig.sqs-redrive-1.85.0.tar.gz.

File metadata

  • Download URL: mbonig.sqs-redrive-1.85.0.tar.gz
  • Upload date:
  • Size: 20.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.6.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.5

File hashes

Hashes for mbonig.sqs-redrive-1.85.0.tar.gz
Algorithm Hash digest
SHA256 1acd5615fc484c8497b11c1ada50ffea21e97c92e8eb1f6798fbd45f78c59252
MD5 1e86c9929ea32509bc0ffaab01714732
BLAKE2b-256 d469ab373bac89389917e8524e8e2e98b92d7021b8440d130ff072d51f3e6e3f

See more details on using hashes here.

File details

Details for the file mbonig.sqs_redrive-1.85.0-py3-none-any.whl.

File metadata

  • Download URL: mbonig.sqs_redrive-1.85.0-py3-none-any.whl
  • Upload date:
  • Size: 21.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.6.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.5

File hashes

Hashes for mbonig.sqs_redrive-1.85.0-py3-none-any.whl
Algorithm Hash digest
SHA256 85e7ef2f197e6b63122db2b14f34f33acd5f5234b4890dbfa38ec4c3c19e7d3f
MD5 57586305f24d35f64b56049ecc64bcf7
BLAKE2b-256 26870d92b42ae2a43fbd7bb6edd6b8d630897b390c3a45944500ab1c081cf88b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page