mypy-boto3-builder 0.7.14

Builder for mypy-boto3.

mypy_boto3

mypy-friendly type annotations for boto3.

Based on boto3_type_annotations by @alliefitter.

• mypy_boto3
  • Installation
    • Build services index manually
  • Latest changes
  • Usage
    • Setup your IDE
    • VSCode
    • PyCharm
    • Other IDEs
    • Explicit type annotations
    • Pylint compatibility
  • How to build
    • Locally
    • With Docker image
  • Differences from boto3-type-annotations
  • Thank you
  • Sub-modules
    • Examples
    • List of all sub-modules

Installation

# install `boto3` type annotations
# for ec2, s3, rds, lambda, sqs, dynamo and cloudformation
# Consumes ~7 MB of space
python -m pip install boto3-stubs[essential]

# or install annotations for services you use
python -m pip install boto3-stubs[acm,apigateway]

Build services index manually

This package generates a few source files depending on services that you installed. Generation is done by a post-install script, so as long as you use pip, pipfile or poetry everything should be done automatically.

However, if you use any other way and notice that services stubs do not work, you can build services index manually.

# Use this command when you add or remove service packages
python -m mypy_boto3

Latest changes

Full changelog can be found in Releases.

Usage

• Install mypy and optionally enable it in your IDE
• Install boto3
• VSCode: Use explicit types for boto3.client, boto3.session.client, client.get_waiter and client.get_paginator calls to enjoy code auto-complete and correct type hints

import boto3

from mypy_boto3 import s3

# you need explicit type annotatins only if your IDE do not support
# function overloads (e.g. VSCode). For PyCharm anf mypy you do not need
# to set types explicitly
client: s3.S3Client = boto3.client("s3")

# IDE autocomplete suggests function name and arguments here
client.create_bucket(Bucket="bucket")

# (mypy) error: Missing positional argument "Key" in call to "get_object" of "S3Client"
client.get_object(Bucket="bucket")

# (mypy) error: Argument "Key" to "get_object" of "S3Client" has incompatible type "None"; expected "str"
client.get_object(Bucket="bucket", Key=None)

resource: s3.S3ServiceResource = boto3.Session(region_name="us-west-1").resource("s3")

# IDE autocomplete suggests function name and arguments here
bucket = resource.Bucket("bucket")

# (mypy) error: Unexpected keyword argument "key" for "upload_file" of "Bucket"; did you mean "Key"?
bucket.upload_file(Filename="my.txt", key="my-txt")

# waiters and paginators are annotated as well
waiter: s3.BucketExistsWaiter = client.get_waiter("bucket_exists")
paginator: s3.ListMultipartUploadsPaginator = client.get_paginator(
    "list_multipart_uploads"
)

Setup your IDE

VSCode

• Install Official Python extension
• Install mypy
• Activate mypy checking in settings: "python.linting.mypyEnabled": true
• Install boto3-stubs with boto3 services you use
• Run python -m mypy_boto3
• Use explicit type annotations because function overload is not fully supported yet

PyCharm

• Install mypy plugin
• Install mypy
• Set path to mypy in mypy plugin settings
• Install boto3-stubs with boto3 services you use
• Run python -m mypy_boto3

Official mypy plugin does not work for some reason for me. If you know how to setup it correctly, please hep me to update this section.

Other IDEs

Please help me to extend this section.

Explicit type annotations

Automatic type discovery is too stressful for PyCharm and does not work in VSCode. So implicit type annotations support has been removed as it is not useful.

To get full advantage of boto3-stubs, you can set types explicitly.

import boto3

from mypy_boto3 import ec2

# covered by boto3-stubs, no explicit type required
session = boto3.session.Session(region_name="us-west-1")

# by default it is botocore.client.BaseClient, but we explicitly
# set it to EC2.EC2Client
ec2_client: ec2.EC2Client = boto3.client("ec2", region_name="us-west-1")

# same for resource
ec2_resource: ec2.EC2ServiceResource = session.resource("ec2")

# PyCharm does not need explicit type annotations here, but VSCode does
bundle_task_complete_waiter: ec2.BundleTaskCompleteWaiter = ec2_client.get_waiter("bundle_task_complete")
describe_volumes_paginator: ec2.DescribeVolumesPaginator = ec2_client.get_paginator("describe_volumes")

# ec2_client, ec2_resource, bundle_task_complete_waiter and describe_volumes_paginator
# now have correct type so IDE autocomplete for methods, arguments and return types
# works as expected. You do not need to specify types explicitly further

Pylint compatibility

It is totally safe to use TYPE_CHECKING flag in order to avoid boto3-stubs dependency in production. However, there is an issue in pylint that it complains about undefined variables. To fix it, set all types to Any in non-TYPE_CHECKING mode.

import boto3
from typing import Any, TYPE_CHECKING

if TYPE_CHECKING:
    from mypy_boto3.ec2 import Client, ServiceResource
    from mypy_boto3.ec2.waiters import BundleTaskCompleteWaiter
    from mypy_boto3.ec2.paginators import DescribeVolumesPaginator
else:
    Client = Any
    ServiceResource = Any
    BundleTaskCompleteWaiter = Any
    DescribeVolumesPaginator = Any

...

How to build

Locally

mypy-boto3 is built for the latest version of boto3. If you need type annotations for another version of boto3, you can use mypy-boto3-builder.

# Install preferred version of `boto3`
python -m pip install boto3==1.10.18 botocore==1.13.18

# Install `mypy-boto3-builder`
python -m pip install mypy-boto3-builder

# Build all packages
# You can specify required services explicitly like
# ./scripts/build.sh -s ec2 s3
./scripts/build.sh

# Install custom `mypy-boto3` packages
./scripts/install.sh

With Docker image

• Install Docker
• Pull latest mypy_boto3_builder version and tag it

docker pull docker.pkg.github.com/vemel/mypy_boto3/mypy_boto3_builder:latest
docker tag docker.pkg.github.com/vemel/mypy_boto3/mypy_boto3_builder:latest mypy_boto3_builder

• Generate stubs in output directory

mkdir output

# generate stubs for all services
docker run -v `pwd`/output:/output -ti mypy_boto3_builder

# generate stubs for s3 service
docker run -v `pwd`/output:/output -ti mypy_boto3_builder -s s3

# generate stubs for a specific boto3 version
docker run -e BOTO3_VERSION=1.10.18 BOTOCORE_VERSION=1.13.18 -v `pwd`/output:/output -ti mypy_boto3_builder

• Install packages from output directory as described above

Differences from boto3-type-annotations

• mypy compatibility
• Fully type annotated
• No need to set types explicitly (depends on your IDE)
• Generated types for return values and arguments
• Added ServiceResource sub-collections
• Support service-specific sub-modules
• Modules documentation
• Type annotations for return structures
• Correct annotations for client.get_waiter and client.get_paginator
• Helper functions for IDEs with no overload support (Hi, VSCode!)

Thank you

• Guys behind boto3-type-annotations, this package is based on top of their work
• black developers for awesome formatting tool
• mypy for doing all dirty work for us

Sub-modules

Examples

You can install any sub-modules using pip

# pip install boto3-stubs[<submodule_name>,...]

# install `boto3` type annotations
# for ec2, s3, rds, lambda, sqs, dynamo and cloudformation
python -m pip install boto3-stubs[essential]

# install ec2, s3 and sqs type annotations
python -m pip install boto3-stubs[s3,ec2,sqs]

# build service index. You should execute this command everytime
# you install or remove service packages
python -m mypy_boto3

List of all sub-modules

• essential - Type annotations for ec2, s3, rds, lambda, sqs, dynamodb and cloudformation services.
• accessanalyzer - Type annotations for boto3 accessanalyzer service.
• acm - Type annotations for boto3 acm service.
• acm-pca - Type annotations for boto3 acm-pca service.
• alexaforbusiness - Type annotations for boto3 alexaforbusiness service.
• amplify - Type annotations for boto3 amplify service.
• apigateway - Type annotations for boto3 apigateway service.
• apigatewaymanagementapi - Type annotations for boto3 apigatewaymanagementapi service.
• apigatewayv2 - Type annotations for boto3 apigatewayv2 service.
• appconfig - Type annotations for boto3 appconfig service.
• application-autoscaling - Type annotations for boto3 application-autoscaling service.
• application-insights - Type annotations for boto3 application-insights service.
• appmesh - Type annotations for boto3 appmesh service.
• appstream - Type annotations for boto3 appstream service.
• appsync - Type annotations for boto3 appsync service.
• athena - Type annotations for boto3 athena service.
• autoscaling - Type annotations for boto3 autoscaling service.
• autoscaling-plans - Type annotations for boto3 autoscaling-plans service.
• backup - Type annotations for boto3 backup service.
• batch - Type annotations for boto3 batch service.
• budgets - Type annotations for boto3 budgets service.
• ce - Type annotations for boto3 ce service.
• chime - Type annotations for boto3 chime service.
• cloud9 - Type annotations for boto3 cloud9 service.
• clouddirectory - Type annotations for boto3 clouddirectory service.
• cloudformation - Type annotations for boto3 cloudformation service.
• cloudfront - Type annotations for boto3 cloudfront service.
• cloudhsm - Type annotations for boto3 cloudhsm service.
• cloudhsmv2 - Type annotations for boto3 cloudhsmv2 service.
• cloudsearch - Type annotations for boto3 cloudsearch service.
• cloudsearchdomain - Type annotations for boto3 cloudsearchdomain service.
• cloudtrail - Type annotations for boto3 cloudtrail service.
• cloudwatch - Type annotations for boto3 cloudwatch service.
• codebuild - Type annotations for boto3 codebuild service.
• codecommit - Type annotations for boto3 codecommit service.
• codedeploy - Type annotations for boto3 codedeploy service.
• codeguru-reviewer - Type annotations for boto3 codeguru-reviewer service.
• codeguruprofiler - Type annotations for boto3 codeguruprofiler service.
• codepipeline - Type annotations for boto3 codepipeline service.
• codestar - Type annotations for boto3 codestar service.
• codestar-notifications - Type annotations for boto3 codestar-notifications service.
• cognito-identity - Type annotations for boto3 cognito-identity service.
• cognito-idp - Type annotations for boto3 cognito-idp service.
• cognito-sync - Type annotations for boto3 cognito-sync service.
• comprehend - Type annotations for boto3 comprehend service.
• comprehendmedical - Type annotations for boto3 comprehendmedical service.
• compute-optimizer - Type annotations for boto3 compute-optimizer service.
• config - Type annotations for boto3 config service.
• connect - Type annotations for boto3 connect service.
• connectparticipant - Type annotations for boto3 connectparticipant service.
• cur - Type annotations for boto3 cur service.
• dataexchange - Type annotations for boto3 dataexchange service.
• datapipeline - Type annotations for boto3 datapipeline service.
• datasync - Type annotations for boto3 datasync service.
• dax - Type annotations for boto3 dax service.
• devicefarm - Type annotations for boto3 devicefarm service.
• directconnect - Type annotations for boto3 directconnect service.
• discovery - Type annotations for boto3 discovery service.
• dlm - Type annotations for boto3 dlm service.
• dms - Type annotations for boto3 dms service.
• docdb - Type annotations for boto3 docdb service.
• ds - Type annotations for boto3 ds service.
• dynamodb - Type annotations for boto3 dynamodb service.
• dynamodbstreams - Type annotations for boto3 dynamodbstreams service.
• ec2 - Type annotations for boto3 ec2 service.
• ec2-instance-connect - Type annotations for boto3 ec2-instance-connect service.
• ecr - Type annotations for boto3 ecr service.
• ecs - Type annotations for boto3 ecs service.
• efs - Type annotations for boto3 efs service.
• eks - Type annotations for boto3 eks service.
• elastic-inference - Type annotations for boto3 elastic-inference service.
• elasticache - Type annotations for boto3 elasticache service.
• elasticbeanstalk - Type annotations for boto3 elasticbeanstalk service.
• elastictranscoder - Type annotations for boto3 elastictranscoder service.
• elb - Type annotations for boto3 elb service.
• elbv2 - Type annotations for boto3 elbv2 service.
• emr - Type annotations for boto3 emr service.
• es - Type annotations for boto3 es service.
• events - Type annotations for boto3 events service.
• firehose - Type annotations for boto3 firehose service.
• fms - Type annotations for boto3 fms service.
• forecast - Type annotations for boto3 forecast service.
• forecastquery - Type annotations for boto3 forecastquery service.
• frauddetector - Type annotations for boto3 frauddetector service.
• fsx - Type annotations for boto3 fsx service.
• gamelift - Type annotations for boto3 gamelift service.
• glacier - Type annotations for boto3 glacier service.
• globalaccelerator - Type annotations for boto3 globalaccelerator service.
• glue - Type annotations for boto3 glue service.
• greengrass - Type annotations for boto3 greengrass service.
• groundstation - Type annotations for boto3 groundstation service.
• guardduty - Type annotations for boto3 guardduty service.
• health - Type annotations for boto3 health service.
• iam - Type annotations for boto3 iam service.
• imagebuilder - Type annotations for boto3 imagebuilder service.
• importexport - Type annotations for boto3 importexport service.
• inspector - Type annotations for boto3 inspector service.
• iot - Type annotations for boto3 iot service.
• iot-data - Type annotations for boto3 iot-data service.
• iot-jobs-data - Type annotations for boto3 iot-jobs-data service.
• iot1click-devices - Type annotations for boto3 iot1click-devices service.
• iot1click-projects - Type annotations for boto3 iot1click-projects service.
• iotanalytics - Type annotations for boto3 iotanalytics service.
• iotevents - Type annotations for boto3 iotevents service.
• iotevents-data - Type annotations for boto3 iotevents-data service.
• iotsecuretunneling - Type annotations for boto3 iotsecuretunneling service.
• iotthingsgraph - Type annotations for boto3 iotthingsgraph service.
• kafka - Type annotations for boto3 kafka service.
• kendra - Type annotations for boto3 kendra service.
• kinesis - Type annotations for boto3 kinesis service.
• kinesis-video-archived-media - Type annotations for boto3 kinesis-video-archived-media service.
• kinesis-video-media - Type annotations for boto3 kinesis-video-media service.
• kinesisanalytics - Type annotations for boto3 kinesisanalytics service.
• kinesisanalyticsv2 - Type annotations for boto3 kinesisanalyticsv2 service.
• kinesisvideo - Type annotations for boto3 kinesisvideo service.
• kms - Type annotations for boto3 kms service.
• lakeformation - Type annotations for boto3 lakeformation service.
• lambda - Type annotations for boto3 lambda service.
• lex-models - Type annotations for boto3 lex-models service.
• lex-runtime - Type annotations for boto3 lex-runtime service.
• license-manager - Type annotations for boto3 license-manager service.
• lightsail - Type annotations for boto3 lightsail service.
• logs - Type annotations for boto3 logs service.
• machinelearning - Type annotations for boto3 machinelearning service.
• macie - Type annotations for boto3 macie service.
• managedblockchain - Type annotations for boto3 managedblockchain service.
• marketplace-catalog - Type annotations for boto3 marketplace-catalog service.
• marketplace-entitlement - Type annotations for boto3 marketplace-entitlement service.
• marketplacecommerceanalytics - Type annotations for boto3 marketplacecommerceanalytics service.
• mediaconnect - Type annotations for boto3 mediaconnect service.
• mediaconvert - Type annotations for boto3 mediaconvert service.
• medialive - Type annotations for boto3 medialive service.
• mediapackage - Type annotations for boto3 mediapackage service.
• mediapackage-vod - Type annotations for boto3 mediapackage-vod service.
• mediastore - Type annotations for boto3 mediastore service.
• mediastore-data - Type annotations for boto3 mediastore-data service.
• mediatailor - Type annotations for boto3 mediatailor service.
• meteringmarketplace - Type annotations for boto3 meteringmarketplace service.
• mgh - Type annotations for boto3 mgh service.
• migrationhub-config - Type annotations for boto3 migrationhub-config service.
• mobile - Type annotations for boto3 mobile service.
• mq - Type annotations for boto3 mq service.
• mturk - Type annotations for boto3 mturk service.
• neptune - Type annotations for boto3 neptune service.
• networkmanager - Type annotations for boto3 networkmanager service.
• opsworks - Type annotations for boto3 opsworks service.
• opsworkscm - Type annotations for boto3 opsworkscm service.
• organizations - Type annotations for boto3 organizations service.
• outposts - Type annotations for boto3 outposts service.
• personalize - Type annotations for boto3 personalize service.
• personalize-events - Type annotations for boto3 personalize-events service.
• personalize-runtime - Type annotations for boto3 personalize-runtime service.
• pi - Type annotations for boto3 pi service.
• pinpoint - Type annotations for boto3 pinpoint service.
• pinpoint-email - Type annotations for boto3 pinpoint-email service.
• pinpoint-sms-voice - Type annotations for boto3 pinpoint-sms-voice service.
• polly - Type annotations for boto3 polly service.
• pricing - Type annotations for boto3 pricing service.
• qldb - Type annotations for boto3 qldb service.
• qldb-session - Type annotations for boto3 qldb-session service.
• quicksight - Type annotations for boto3 quicksight service.
• ram - Type annotations for boto3 ram service.
• rds - Type annotations for boto3 rds service.
• rds-data - Type annotations for boto3 rds-data service.
• redshift - Type annotations for boto3 redshift service.
• rekognition - Type annotations for boto3 rekognition service.
• resource-groups - Type annotations for boto3 resource-groups service.
• resourcegroupstaggingapi - Type annotations for boto3 resourcegroupstaggingapi service.
• robomaker - Type annotations for boto3 robomaker service.
• route53 - Type annotations for boto3 route53 service.
• route53domains - Type annotations for boto3 route53domains service.
• route53resolver - Type annotations for boto3 route53resolver service.
• s3 - Type annotations for boto3 s3 service.
• s3control - Type annotations for boto3 s3control service.
• sagemaker - Type annotations for boto3 sagemaker service.
• sagemaker-a2i-runtime - Type annotations for boto3 sagemaker-a2i-runtime service.
• sagemaker-runtime - Type annotations for boto3 sagemaker-runtime service.
• savingsplans - Type annotations for boto3 savingsplans service.
• schemas - Type annotations for boto3 schemas service.
• sdb - Type annotations for boto3 sdb service.
• secretsmanager - Type annotations for boto3 secretsmanager service.
• securityhub - Type annotations for boto3 securityhub service.
• serverlessrepo - Type annotations for boto3 serverlessrepo service.
• service-quotas - Type annotations for boto3 service-quotas service.
• servicecatalog - Type annotations for boto3 servicecatalog service.
• servicediscovery - Type annotations for boto3 servicediscovery service.
• ses - Type annotations for boto3 ses service.
• sesv2 - Type annotations for boto3 sesv2 service.
• shield - Type annotations for boto3 shield service.
• signer - Type annotations for boto3 signer service.
• sms - Type annotations for boto3 sms service.
• sms-voice - Type annotations for boto3 sms-voice service.
• snowball - Type annotations for boto3 snowball service.
• sns - Type annotations for boto3 sns service.
• sqs - Type annotations for boto3 sqs service.
• ssm - Type annotations for boto3 ssm service.
• sso - Type annotations for boto3 sso service.
• sso-oidc - Type annotations for boto3 sso-oidc service.
• stepfunctions - Type annotations for boto3 stepfunctions service.
• storagegateway - Type annotations for boto3 storagegateway service.
• sts - Type annotations for boto3 sts service.
• support - Type annotations for boto3 support service.
• swf - Type annotations for boto3 swf service.
• textract - Type annotations for boto3 textract service.
• transcribe - Type annotations for boto3 transcribe service.
• transfer - Type annotations for boto3 transfer service.
• translate - Type annotations for boto3 translate service.
• waf - Type annotations for boto3 waf service.
• waf-regional - Type annotations for boto3 waf-regional service.
• wafv2 - Type annotations for boto3 wafv2 service.
• workdocs - Type annotations for boto3 workdocs service.
• worklink - Type annotations for boto3 worklink service.
• workmail - Type annotations for boto3 workmail service.
• workmailmessageflow - Type annotations for boto3 workmailmessageflow service.
• workspaces - Type annotations for boto3 workspaces service.
• xray - Type annotations for boto3 xray service.