Skip to main content

Python test fixtures for your local AWS cloud stack

Project description

boto3-fixtures

PyPI version TravisCI build status Code Coverage

boto3-fixtures provides test fixtures for your local AWS cloud stack.

Testing software which touches cloud infrastructure doesn't have to be difficult! boto3-fixtures provides a dead-simple framework for setup+teardown of mocked AWS infrastructure. Use in combination with projects like moto or localstack.

Supports

  • Kinesis
  • SQS
  • SNS
  • S3
  • Lambda
  • DynamoDB

Please submit a PR or issue if you'd like to see support for a specific AWS service!

Getting Started

This library provides a context decorator, boto3_fixtures.Service, which will setup and teardown AWS services.

import boto3_fixtures

with boto3_fixtures.Service("sqs", queues=["my-queue"]) as svc:
    # Queues exist
    for queue in svc.state["queues"]:
      print(f"{queue.name} - {queue.arn} - {queue.url}")

# Queues destroyed

Combine this with a local testing stack of your choice (moto, localstack).

import boto3_fixtures, moto

with moto.mock_sqs():
    with boto3_fixtures.Service("sqs", queues=["first-queue", "second-queue"]) as svc:
      # ...

Generating Pytest Fixtures

To make your life even easier, we've boiled all of the above down into pytest fixture generators.

import boto3_fixtures as b3f

aws = b3f.contrib.pytest.moto_fixture(
  services=["dynamodb", "kinesis", "sqs", "s3", "lambda"],
  scope="class",
)

sqs = b3f.contrib.pytest.service_fixture("sqs", scope="class", queues=fixtures.SQS)
kinesis = b3f.contrib.pytest.service_fixture("kinesis", scope="class", streams=fixtures.KINESIS)
dynamodb = b3f.contrib.pytest.service_fixture("dynamodb", scope="class", tables=fixtures.DYNAMODB)
s3 = b3f.contrib.pytest.service_fixture("s3", scope="class", buckets=fixtures.S3)
lam = b3f.contrib.pytest.service_fixture("lambda", scope="class", lambdas=fixtures.LAMBDA)
sns = b3f.contrib.pytest.service_fixture("sns", scope="class", topics=fixtures.TOPICS)


# Example Usage
def test_my_code(sqs):
    boto3.client("sqs").list_queues()

The aws() fixture

To ensure your mocked cloud is a dependency of your service fixtures, boto3-fixtures expects you to create a fixture named aws. If you don't take advantage of this, your local cloud stack may be torn down before your service, leading to boto3 exceptions when tearing down the services.

# Example: localstack via pytest-localstack
import pytest_localstack

aws = pytest_localstack.patch_fixture(
  services=["sqs"],
  scope="class",
)

# Example: moto via boto3-fixtures
import boto3_fixtures as b3f

aws = b3f.contrib.pytest.moto_fixture(
  services=["sqs"],
  scope="class",
)

Configuring Services

Configuration of a service may be either a list of names List[str] or a list of configs List[dict] containing boto3 parameters.

Service List of Names List of Configs
s3 yes yes
sqs yes yes
kinesis yes yes
dynamodb yes
lambda yes
sns yes yes

For example, your configuration might look like this:

S3 = ["first-bucket", "second-bucket"]

SQS = ["first-queue", "second-queue"]

KINESIS = ["first-stream", "second-stream"]

DYNAMODB = [
    {
        "AttributeDefinitions": [
            {"AttributeName": "uri", "AttributeType": "S"},
            {"AttributeName": "timestamp", "AttributeType": "S"},
        ],
        "TableName": "test-dbd-table",
        "KeySchema": [
            {"AttributeName": "uri", "KeyType": "HASH"},
            {"AttributeName": "timestamp", "KeyType": "RANGE"},
        ],
    }
]

LAMBDA = [
    {
        "zip_path": "dist/build.zip",
        "FunctionName": "my_lambda",
        "Runtime": "python3.6",
        "Environment": {"foo": True},
    }
]

SNS = [
    "my-topic-with-default-attrs",
    {
        "Name": "my-topic-with-additional-params",
        "Tags": [{"Key": "key1", "Value": "val1"}],
        "Attributes": {
            "DisplayName": "YourSystemIsOnFireTopic",
        },
    }
]

These configurations don't have to be static. You could use a pytest fixture to build or compile a list of resources that you want mocked.

Using both moto and localstack

You can point the aws fixture at moto or localstack to explicitly to require a specific stack to exist for the duration of your service fixture. For example, if you use both stacks:

# conftest.py
stack_config = {
    "services": ["dynamodb", "kinesis", "sqs", "s3", "lambda"],
    "scope": "class",
    "autouse": False,
    "region_name": "us-east-1",
}

localstack = pytest_localstack.patch_fixture(**stack_config)
moto = b3f.contrib.pytest.moto_fixture(**stack_config)

@pytest.fixture(scope="class")
def aws(moto):
    pass

# component/conftest.py
@pytest.fixture(scope="class")
def aws(localstack):
    pass

Known Issues

  • Using both pytest-localstack and moto in the same project may break if the pytest-localstack tests run first. It's suspected this is due to an issue with cleanup with the pytest-localstack session, but is this is still under investigation.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

boto3-fixtures-0.0.8.tar.gz (12.4 kB view hashes)

Uploaded source

Built Distribution

boto3_fixtures-0.0.8-py2.py3-none-any.whl (16.6 kB view hashes)

Uploaded py2 py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page