Skip to main content

Amazon SQS Extended Client Library for Python for sending large payloads that exceed sqs limitations via S3

Project description

Boto3 SQS Extended Client Library for Python

Build Status codecov

The Amazon SQS Extended Client Library for Python has been modelled after the original Amazon SQS Extended client library This python library enables you to manage Amazon SQS message payloads with Amazon S3. This is especially useful for storing and retrieving messages with a message payload size greater than the current SQS limit of 256 KB, up to a maximum of 2 GB. Specifically, you can use this library to:

  • Specify whether message payloads are always stored in Amazon S3 or only when a message's size exceeds 256 KB.

  • Send a message that references a single message object stored in an Amazon S3 bucket.

  • Get the corresponding message object from an Amazon S3 bucket.

  • Delete the corresponding message object from an Amazon S3 bucket.

Getting Started

  • Sign up for AWS -- Before you begin, you need an AWS account. For more information about creating an AWS account and retrieving your AWS credentials, see [AWS Account and Credentials.

  • Sign up for Amazon SQS -- Go to the Amazon SQS console to sign up for the service.

  • Minimum requirements -- To use the sample application, you'll need python 3 (and above)

  • Further information - Read the API documentation.


    import base64

    from pysqs_extended_client.SQSClientExtended import SQSClientExtended
    # from string import ascii_letters, digits
    # from random import choice
    from pysqs_extended_client.config import (AWS_SQS_QUEUE_URL, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION)
    if __name__ == '__main__':
        sqs = SQSClientExtended(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION, 'tiptapcode-sqs-data')
        # _100mb_large_string = ''.join([choice(ascii_letters + digits) for i in range(104857600)])
        # message = "_100mb_large_string"
        message = None
        with open("C:\\DjangoCourse\\Courses\\celery\\introduction-promo.mp4", "rb") as image_file:
            encoded_string = base64.b64encode(
            message = encoded_string.decode("utf-8")
        sqs.send_message(AWS_SQS_QUEUE_URL, message)
        res = sqs.receive_message(AWS_SQS_QUEUE_URL)
        for message in res:
            sqs.delete_message(AWS_SQS_QUEUE_URL, message.get('ReceiptHandle'))


  • Give feedback here.
  • If you'd like to contribute a new feature or bug fix, go ahead submit a pull request.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pysqs-extended-client-0.0.1.tar.gz (7.0 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page