Tweet Queue for AWS CDK
This is an AWS CDK construct library which allows you to get a feed of Twitter search results into an SQS queue. It works by periodically polling the freely available Twitter Standard Search API and sending all new tweets to an SQS queue.
Inspired by @jlhood's aws-serverless-twitter-event-source
Architecture
- A CloudWatch Event Rule triggers the poller AWS Lambda function periodically
- The poller reads the last checkpoint from a DynamoDB table (if exists)
- The poller issues a Twitter search query for all new tweets
- The poller enqueues all tweets to an SQS queue
- The poller stores the ID of the last tweet into the DynamoDB checkpoint table.
- Rinse & repeat.
Twitter API Keys
To issue a Twitter search request, you will need to apply for a Twitter developer account, and obtain API keys through by defining a new application.
The Twitter API keys are read by the poller from an AWS Secrets
Manager entry. The entry must contain
the following attributes: consumer_key, consumer_secret, access_token_key
and access_token_secret (exact names).
- Create a new AWS Secrets Manager entry for your API keys
- Fill in the key values as shown below:
- Store the key
- Obtain the ARN of the secret (you will need it soon).
Usage
Use npm to install the module in your CDK project. This will also add it to
your package.json file.
$ npm install cdk-tweet-queue
Add a TweetQueue to your CDK stack:
from cdk_tweet_queue import TweetQueue
queue = TweetQueue(self, "TweetStream",
# this is the ARN of the secret you stored
secret_arn="arn:aws:secretsmanager:us-east-1:1234567891234:secret:xxxxxxxxx",
# twitter search query
# see https://developer.twitter.com/en/docs/tweets/search/guides/standard-operators
query="#awscdk",
# optional properties
interval_min=60, # optional: polling interval in minutes
retention_period_sec=60, # optional: queue retention period
visibility_timeout_sec=60
)
Now, queue is an sqs.Queue object and can be used anywhere a queue is
accepted. For example, you could process the queue messages using an AWS Lambda
function by setting up an SQS event source mapping.
Development
The project is managed by projen and offers the following commands:
yarn projen- Synthesize the project configuration.yarn compile- Compile all source code.yarn test- Run all tests.yarn build- Complie, test, and package the module.
Integration test
There is also an integration test that can be executed by running the following commands. You will need to set the TWEET_QUEUE_SECRET_ARN environment variable in order for the test to be able to use your Twitter API keys.
$ yarn integ:deploy
Don't forget to destroy:
$ yarn integ:destroy
You can also run any cdk command on the integration test application by running:
yarn integ <command>
License
Apache-2.0
2 maintainers
Metadata
2 maintainers
Release files for cdk-tweet-queue 2.0.964
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| cdk_tweet_queue-2.0.964.tar.gz | 123.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| cdk_tweet_queue-2.0.964-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 245.1 kB
Release files / cdk_tweet_queue-2.0.964.tar.gz
| Download URL | cdk_tweet_queue-2.0.964.tar.gz |
|---|---|
| Size | 123.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8c7fd1e898cab31c90f15e617dfa7da4972aea59f7d1b410fac680350d825c31
|
|
BLAKE2b-256 checksum How to use checksums |
c926ad2d334e0611f540e27f863e603eee6a3d7643478206ea5d05bf01a1391d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.14.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 28, 2026.
Transparency logRelease files / cdk_tweet_queue-2.0.964-py3-none-any.whl
| Download URL | cdk_tweet_queue-2.0.964-py3-none-any.whl |
|---|---|
| Size | 121.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
797f2c99e1ea120fd05e009b8caf84a950e95357b920ea8f192f0c0f9ff39b6d
|
|
BLAKE2b-256 checksum How to use checksums |
f77c885477d41158d04866f1454aa8cadace10b1d427f972372af071f647ac0a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.14.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 28, 2026.
Transparency log
