Skip to main content

Async boto3 wrapper

Project description

Async AWS SDK for Python

https://img.shields.io/pypi/v/aioboto3.svg https://img.shields.io/travis/terrycain/aioboto3.svg Documentation Status Updates

aiobotocore has been copied into this project, and updated until https://github.com/aio-libs/aiobotocore/pull/639 is merged

This package is mostly just a wrapper combining the great work of boto3 and aiobotocore.

aiobotocore allows you to use near enough all of the boto3 client commands in an async manner just by prefixing the command with await.

With aioboto3 you can now use the higher level APIs provided by boto3 in an asynchronous manner. Mainly I developed this as I wanted to use the boto3 dynamodb Table object in some async microservices.

While all resources in boto3 should work I havent tested them all, so if what your after is not in the table below then try it out, if it works drop me an issue with a simple test case and I’ll add it to the table.

Services

Status

DynamoDB Service Resource

Tested and working

DynamoDB Table

Tested and working

S3

Working

Kinesis

Working

SSM Parameter Store

Working

Athena

Working

Example

Simple example of using aioboto3 to put items into a dynamodb table

import asyncio
import aioboto3
from boto3.dynamodb.conditions import Key


async def main():
    async with aioboto3.resource('dynamodb', region_name='eu-central-1') as dynamo_resource:
        table = dynamo_resource.Table('test_table')

        await table.put_item(
            Item={'pk': 'test1', 'col1': 'some_data'}
        )

        result = await table.query(
            KeyConditionExpression=Key('pk').eq('test1')
        )

        print(result['Items'])

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

# Outputs:
#  [{'col1': 'some_data', 'pk': 'test1'}]

Things that either dont work or have been patched

As this library literally wraps boto3, its inevitable that some things won’t magically be async.

  • s3_client.copy This is performed by the s3transfer module. I believe s3_client.copy_object performs the same function

Fixed:

  • s3_client.download_file* This is performed by the s3transfer module. – Patched with get_object

  • s3_client.upload_file* This is performed by the s3transfer module. – Patched with custom multipart upload

  • dynamodb_resource.Table.batch_writer This now returns an async context manager which performs the same function

Documentation

Docs are here - https://aioboto3.readthedocs.io/en/latest/

Examples here - https://aioboto3.readthedocs.io/en/latest/usage.html

Features

  • Closely mimics the usage of boto3.

Todo

  • More Examples

  • Set up docs

  • Look into monkey-patching the aws xray sdk to be more async if it needs to be.

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template. It also makes use of the aiobotocore and boto3 libraries. All the credit goes to them, this is mainly a wrapper with some examples.

History

5.0.0 (2018-10-12)

  • Updated lots of dependencies

  • Changed s3.upload_fileobj from using put_object to doing a multipart upload

  • Created s3.copy shim that runs get_object then does multipart upload, could do with a better implementation though.

4.1.2 (2018-08-28)

  • updated pypi credentials

4.1.0 (2018-08-28)

  • aiobotocore dependancy bump

4.0.2 (2018-08-03)

  • Dependancy bump

4.0.0 (2018-05-09)

  • Dependancy bump

  • Now using aiobotocore 0.8.0

  • Dropped < py3.5 support

  • Now using async def / await syntax

  • Fixed boto3 dependancy so it only uses a boto3 version supported by aiobotocore’s max botocore dependancy

  • Important, `__call__` in `AIOServiceAction` tries to yield from a coroutine in a non-coroutine, this code shouldn’t be hit anymore but I can’t guarantee that, so instead `__call__` was duplicated and awaited properly so “should” be fine. Credit goes to Arnulfo Solis for doing PR.

3.0.0 (2018-03-29)

  • Dependancy bump

  • Asyncified dynamodb Table Batch Writer + Tests

  • Added batch writer examples

  • Now using aiobotocore 0.6.0

2.2.0 (2018-01-24)

  • Dependancy bump

2.1.0 (2018-01-23)

  • Dependancy bump

  • Fix bug where extras isn’t packaged

2.0.0 (2017-12-30)

  • Patched most s3transfer functions

1.1.2 (2017-11-29)

  • Fixup of lingering GPL license texts

0.1.0 (2017-09-25)

  • First release on PyPI.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aioboto3-6.0.0.tar.gz (40.9 kB view details)

Uploaded Source

Built Distribution

aioboto3-6.0.0-py2.py3-none-any.whl (33.1 kB view details)

Uploaded Python 2Python 3

File details

Details for the file aioboto3-6.0.0.tar.gz.

File metadata

  • Download URL: aioboto3-6.0.0.tar.gz
  • Upload date:
  • Size: 40.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.1 setuptools/40.6.2 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.3

File hashes

Hashes for aioboto3-6.0.0.tar.gz
Algorithm Hash digest
SHA256 238431f94dcccfab65dd747446890f5ad122784bc302d6b01ae74041d7eaf5cc
MD5 426c8cc2790e8f356278fc825063cf37
BLAKE2b-256 3fea5bd5d246cb0da3148de8f5a299230fdbd505460b6723418aec4245bb88da

See more details on using hashes here.

File details

Details for the file aioboto3-6.0.0-py2.py3-none-any.whl.

File metadata

  • Download URL: aioboto3-6.0.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 33.1 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.1 setuptools/40.6.2 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.3

File hashes

Hashes for aioboto3-6.0.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 24240812b0f31c61e5078f1187b9dc2ece5fb1f00f7a3ea94a8051a366e88988
MD5 3c30d260cff734925bcb70f8693b5ee2
BLAKE2b-256 665d99a7c6e08489f42f36eb2113c761d6244e8fbb3547548e07facacfccc703

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page