Skip to main content

Tweet Queue for AWS CDK

This is an AWS CDK construct library which allows you to get a feed of Twitter search results into an SQS queue. It works by periodically polling the freely available Twitter Standard Search API and sending all new tweets to an SQS queue.

Inspired by @jlhood's aws-serverless-twitter-event-source

Architecture

  1. A CloudWatch Event Rule triggers the poller AWS Lambda function periodically
  2. The poller reads the last checkpoint from a DynamoDB table (if exists)
  3. The poller issues a Twitter search query for all new tweets
  4. The poller enqueues all tweets to an SQS queue
  5. The poller stores the ID of the last tweet into the DynamoDB checkpoint table.
  6. Rinse & repeat.

Twitter API Keys

To issue a Twitter search request, you will need to apply for a Twitter developer account, and obtain API keys through by defining a new application.

The Twitter API keys are read by the poller from an AWS Secrets Manager entry. The entry must contain the following attributes: consumer_key, consumer_secret, access_token_key and access_token_secret (exact names).

  1. Create a new AWS Secrets Manager entry for your API keys
  2. Fill in the key values as shown below:
  3. Store the key
  4. Obtain the ARN of the secret (you will need it soon).

Usage

Use npm to install the module in your CDK project. This will also add it to your package.json file.

$ npm install cdk-tweet-queue

Add a TweetQueue to your CDK stack:

from cdk_tweet_queue import TweetQueue


queue = TweetQueue(self, "TweetStream",
    # this is the ARN of the secret you stored
    secret_arn="arn:aws:secretsmanager:us-east-1:1234567891234:secret:xxxxxxxxx",

    # twitter search query
    # see https://developer.twitter.com/en/docs/tweets/search/guides/standard-operators
    query="#awscdk",

    # optional properties
    interval_min=60,  # optional: polling interval in minutes
    retention_period_sec=60,  # optional: queue retention period
    visibility_timeout_sec=60
)

Now, queue is an sqs.Queue object and can be used anywhere a queue is accepted. For example, you could process the queue messages using an AWS Lambda function by setting up an SQS event source mapping.

Development

The project is managed by projen and offers the following commands:

  • yarn projen - Synthesize the project configuration.
  • yarn compile - Compile all source code.
  • yarn test - Run all tests.
  • yarn build - Complie, test, and package the module.

Integration test

There is also an integration test that can be executed by running the following commands. You will need to set the TWEET_QUEUE_SECRET_ARN environment variable in order for the test to be able to use your Twitter API keys.

$ yarn integ:deploy

Don't forget to destroy:

$ yarn integ:destroy

You can also run any cdk command on the integration test application by running:

yarn integ <command>

License

Apache-2.0

Metadata

Release files for cdk-tweet-queue 2.0.969

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cdk-tweet-queue 2.0.969
File Size Uploaded Provenance Actions
cdk_tweet_queue-2.0.969.tar.gz 123.9 kB With provenance Details for cdk_tweet_queue-2.0.969.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for cdk-tweet-queue 2.0.969
File Interpreter ABI Platform Actions
cdk_tweet_queue-2.0.969-py3-none-any.whl Python 3 none any Details for cdk_tweet_queue-2.0.969-py3-none-any.whl

Total release size: 245.9 kB

Release files / cdk_tweet_queue-2.0.969.tar.gz

Download URL cdk_tweet_queue-2.0.969.tar.gz
Size 123.9 kB
Tags Source
SHA-256 checksum
How to use checksums
fe2364f0d99f164be66916636ee8d69fbd307a505bdc8ac823762721a8ad9a60
BLAKE2b-256 checksum
How to use checksums
5992db44f6337dd22cc043081a2791755bb2aca9b4538a762d659d76ab82c149
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.15.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

Release files / cdk_tweet_queue-2.0.969-py3-none-any.whl

Download URL cdk_tweet_queue-2.0.969-py3-none-any.whl
Size 122.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
37af79af8b3d8d353ecfffc2dd3e320c2a31e19e4a38ba9954089b1483b91de1
BLAKE2b-256 checksum
How to use checksums
09035dedd9d3e565ed8747d9c85b77d075762fc46eab5a28e73491a4aab01772
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.15.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

Security for cdk-tweet-queue 2.0.969

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for cdk_tweet_queue-2.0.969.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

2.0.969 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page