Skip to main content

Polls AWS SQS messages and executes your callback.

Project description

sqs-polling

sqs-polling is simple AWS SQS daemon. The following is all.

  • Polls SQS at regular intervals.
  • Receives messages and executes your callback.
  • If your callback function returns True, deletes the message.

Installation

$ pip install sqs-polling

Usage

The key function is sqs_polling, please check its interface.

The simplest usage is below.

from sqs_polling import sqs_polling


def your_callback(message, greeting):
    print(greeting + message["name"])
    return True


your_queue_url="https://sqs.your-region.amazonaws.com/XXXXXXXXXXXX/your-sqs",
sqs_polling(queue_url=your_queue_url, callback=your_callback, callback_args={"greeting": "Hello, "})

Callback

Your callback function must return 1 bool value. If True, deletes the processed message. If False, the message is remaining in your SQS and reprocessed after visibility_timeout seconds. It is recommended to return False if message processing fails.

Configuration

AWS account profile

In the example above, explicit AWS account profiles is omitted.

If explicit authentication is required, you can access SQS by setting additional profiles in aws_profile_dict argument.

your_aws_profile = {
    "region_name": "your-region",
    "aws_access_key_id": "XXXXXXXXXXXXXXXXXXXX",
    "aws_secret_access_key": "YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY",
}

sqs_polling(queue_url=your_queue_url, callback=your_callback, aws_profile_dict=your_aws_profile)

Polling

The arguments related to polling settings are as follows.

  • extract_body: Whether to extract the message body. (default: True)
    • The body here refers to $.Messages.Body JSON element of SQS messages.
  • interval_seconds: Message reception interval seconds. (default: 1.0)
  • visibility_timeout: Message visibility timeout seconds. (default: 300)

Concurrency

sqs-polling can process messages in parallel using threads or processes. It can be controlled by the following arguments.

  • max_workers: Max number of concurrent threads or processes. (default: 1)
  • process_worker: If this is True, processes is used instead of threads. (default: False)
# Parallel processing with up to 64 threads.
sqs_polling(queue_url=your_queue_url, callback=your_callback, max_workers=64)

# Parallel processing with up to 4 processes.
sqs_polling(queue_url=your_queue_url, callback=your_callback, process_worker=True, max_workers=4)

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for sqs-polling, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size sqs_polling-0.1.0-py3-none-any.whl (5.7 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size sqs-polling-0.1.0.tar.gz (4.8 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page