Skip to main content

Tweet Queue for AWS CDK

This is an AWS CDK construct library which allows you to get a feed of Twitter search results into an SQS queue. It works by periodically polling the freely available Twitter Standard Search API and sending all new tweets to an SQS queue.

Inspired by @jlhood's aws-serverless-twitter-event-source

Architecture

  1. A CloudWatch Event Rule triggers the poller AWS Lambda function periodically
  2. The poller reads the last checkpoint from a DynamoDB table (if exists)
  3. The poller issues a Twitter search query for all new tweets
  4. The poller enqueues all tweets to an SQS queue
  5. The poller stores the ID of the last tweet into the DynamoDB checkpoint table.
  6. Rinse & repeat.

Twitter API Keys

To issue a Twitter search request, you will need to apply for a Twitter developer account, and obtain API keys through by defining a new application.

The Twitter API keys are read by the poller from an AWS Secrets Manager entry. The entry must contain the following attributes: consumer_key, consumer_secret, access_token_key and access_token_secret (exact names).

  1. Create a new AWS Secrets Manager entry for your API keys
  2. Fill in the key values as shown below:
  3. Store the key
  4. Obtain the ARN of the secret (you will need it soon).

Usage

Use npm to install the module in your CDK project. This will also add it to your package.json file.

$ npm install cdk-tweet-queue

Add a TweetQueue to your CDK stack:

from cdk_tweet_queue import TweetQueue


queue = TweetQueue(self, "TweetStream",
    # this is the ARN of the secret you stored
    secret_arn="arn:aws:secretsmanager:us-east-1:1234567891234:secret:xxxxxxxxx",

    # twitter search query
    # see https://developer.twitter.com/en/docs/tweets/search/guides/standard-operators
    query="#awscdk",

    # optional properties
    interval_min=60,  # optional: polling interval in minutes
    retention_period_sec=60,  # optional: queue retention period
    visibility_timeout_sec=60
)

Now, queue is an sqs.Queue object and can be used anywhere a queue is accepted. For example, you could process the queue messages using an AWS Lambda function by setting up an SQS event source mapping.

Development

The project is managed by projen and offers the following commands:

  • yarn projen - Synthesize the project configuration.
  • yarn compile - Compile all source code.
  • yarn test - Run all tests.
  • yarn build - Complie, test, and package the module.

Integration test

There is also an integration test that can be executed by running the following commands. You will need to set the TWEET_QUEUE_SECRET_ARN environment variable in order for the test to be able to use your Twitter API keys.

$ yarn integ:deploy

Don't forget to destroy:

$ yarn integ:destroy

You can also run any cdk command on the integration test application by running:

yarn integ <command>

License

Apache-2.0

Metadata

Release files for cdk-tweet-queue 2.0.965

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cdk-tweet-queue 2.0.965
File Size Uploaded Provenance Actions
cdk_tweet_queue-2.0.965.tar.gz 123.6 kB With provenance Details for cdk_tweet_queue-2.0.965.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for cdk-tweet-queue 2.0.965
File Interpreter ABI Platform Actions
cdk_tweet_queue-2.0.965-py3-none-any.whl Python 3 none any Details for cdk_tweet_queue-2.0.965-py3-none-any.whl

Total release size: 245.5 kB

Release files / cdk_tweet_queue-2.0.965.tar.gz

Download URL cdk_tweet_queue-2.0.965.tar.gz
Size 123.6 kB
Tags Source
SHA-256 checksum
How to use checksums
b1b1dae89559ed9fa70532893454210074611e166c00e4d9473ec0cfc26b5afd
BLAKE2b-256 checksum
How to use checksums
552e50e6d59c8664dd9e280264edff463bba4c7b713c53827ce8ed2effc4d7d7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.14.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 12, 2026.

Transparency log

Release files / cdk_tweet_queue-2.0.965-py3-none-any.whl

Download URL cdk_tweet_queue-2.0.965-py3-none-any.whl
Size 121.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
9e18855bf60b80cc259a72dea75deda1c295f9cbe6befc008811fb0a040eca46
BLAKE2b-256 checksum
How to use checksums
c096da2e097dccf27149bacb0004e6ba015f78b557194841781d0c3d6d016999
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.14.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 12, 2026.

Transparency log

Security for cdk-tweet-queue 2.0.965

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for cdk_tweet_queue-2.0.965.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 12, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

2.0.965 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page