Skip to main content

Polls AWS SQS messages and executes your callback.

Project description

sqs-polling

sqs-polling is simple AWS SQS daemon. The following is all.

  • Polls SQS at regular intervals.
  • Receives messages and executes your callback.
  • If your callback function returns True, deletes the message.

Installation

$ pip install sqs-polling

Usage

The key function is sqs_polling, please check its interface.

The simplest usage is below.

from sqs_polling import sqs_polling


def your_callback(message, greeting):
    print(greeting + message["name"])
    return True


your_queue_url="https://sqs.your-region.amazonaws.com/XXXXXXXXXXXX/your-sqs",
sqs_polling(queue_url=your_queue_url, callback=your_callback, callback_args={"greeting": "Hello, "})

Callback

Your callback function must return 1 bool value. If True, deletes the processed message. If False, the message is remaining in your SQS and reprocessed after visibility_timeout seconds. It is recommended to return False if message processing fails.

Configuration

AWS account profile

In the example above, explicit AWS account profiles is omitted.

If explicit authentication is required, you can access SQS by setting additional profiles in aws_profile_dict argument.

your_aws_profile = {
    "region_name": "your-region",
    "aws_access_key_id": "XXXXXXXXXXXXXXXXXXXX",
    "aws_secret_access_key": "YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY",
}

sqs_polling(queue_url=your_queue_url, callback=your_callback, aws_profile_dict=your_aws_profile)

Polling

The arguments related to polling settings are as follows.

  • extract_body: Whether to extract the message body. (default: True)
    • The body here refers to $.Messages.Body JSON element of SQS messages.
  • interval_seconds: Message reception interval seconds. (default: 1.0)
  • visibility_timeout: Message visibility timeout seconds. (default: 300)

Concurrency

sqs-polling can process messages in parallel using threads or processes. It can be controlled by the following arguments.

  • max_workers: Max number of concurrent threads or processes. (default: 1)
  • process_worker: If this is True, processes is used instead of threads. (default: False)
# Parallel processing with up to 64 threads.
sqs_polling(queue_url=your_queue_url, callback=your_callback, max_workers=64)

# Parallel processing with up to 4 processes.
sqs_polling(queue_url=your_queue_url, callback=your_callback, process_worker=True, max_workers=4)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sqs-polling-0.1.0.tar.gz (4.8 kB view details)

Uploaded Source

Built Distribution

sqs_polling-0.1.0-py3-none-any.whl (5.7 kB view details)

Uploaded Python 3

File details

Details for the file sqs-polling-0.1.0.tar.gz.

File metadata

  • Download URL: sqs-polling-0.1.0.tar.gz
  • Upload date:
  • Size: 4.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.5.7

File hashes

Hashes for sqs-polling-0.1.0.tar.gz
Algorithm Hash digest
SHA256 0c3c6f40255fc3d1a94e51b736ffc50f669db72d0e1ec9f5feba1cee2cae2985
MD5 1ad3ab1cb0e7d01941bd4357e24d8e22
BLAKE2b-256 78c1dcefd372e9df04c69f53ace45f0a7746b01a0b39278d4cfadd8bafa26f72

See more details on using hashes here.

File details

Details for the file sqs_polling-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: sqs_polling-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 5.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.5.7

File hashes

Hashes for sqs_polling-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 9855305d92520aa283abc62d909a8bd45473e18db12b242db4d1686a25628841
MD5 812e8174565e4ae94773bd17946a26fa
BLAKE2b-256 4c15b31ad964a85fb88a0a26d388dae5585eb8c1f12e227f28286b17c03aca1b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page