boto3-fixtures 0.0.8

Python test fixtures for your local AWS cloud stack

boto3-fixtures

boto3-fixtures provides test fixtures for your local AWS cloud stack.

Testing software which touches cloud infrastructure doesn't have to be difficult! boto3-fixtures provides a dead-simple framework for setup+teardown of mocked AWS infrastructure. Use in combination with projects like moto or localstack.

Supports

• Kinesis
• SQS
• SNS
• S3
• Lambda
• DynamoDB

Please submit a PR or issue if you'd like to see support for a specific AWS service!

Getting Started

This library provides a context decorator, boto3_fixtures.Service, which will setup and teardown AWS services.

import boto3_fixtures

with boto3_fixtures.Service("sqs", queues=["my-queue"]) as svc:
    # Queues exist
    for queue in svc.state["queues"]:
      print(f"{queue.name} - {queue.arn} - {queue.url}")

# Queues destroyed

Combine this with a local testing stack of your choice (moto, localstack).

import boto3_fixtures, moto

with moto.mock_sqs():
    with boto3_fixtures.Service("sqs", queues=["first-queue", "second-queue"]) as svc:
      # ...

Generating Pytest Fixtures

To make your life even easier, we've boiled all of the above down into pytest fixture generators.

import boto3_fixtures as b3f

aws = b3f.contrib.pytest.moto_fixture(
  services=["dynamodb", "kinesis", "sqs", "s3", "lambda"],
  scope="class",
)

sqs = b3f.contrib.pytest.service_fixture("sqs", scope="class", queues=fixtures.SQS)
kinesis = b3f.contrib.pytest.service_fixture("kinesis", scope="class", streams=fixtures.KINESIS)
dynamodb = b3f.contrib.pytest.service_fixture("dynamodb", scope="class", tables=fixtures.DYNAMODB)
s3 = b3f.contrib.pytest.service_fixture("s3", scope="class", buckets=fixtures.S3)
lam = b3f.contrib.pytest.service_fixture("lambda", scope="class", lambdas=fixtures.LAMBDA)
sns = b3f.contrib.pytest.service_fixture("sns", scope="class", topics=fixtures.TOPICS)


# Example Usage
def test_my_code(sqs):
    boto3.client("sqs").list_queues()

The aws() fixture

To ensure your mocked cloud is a dependency of your service fixtures, boto3-fixtures expects you to create a fixture named aws. If you don't take advantage of this, your local cloud stack may be torn down before your service, leading to boto3 exceptions when tearing down the services.

# Example: localstack via pytest-localstack
import pytest_localstack

aws = pytest_localstack.patch_fixture(
  services=["sqs"],
  scope="class",
)

# Example: moto via boto3-fixtures
import boto3_fixtures as b3f

aws = b3f.contrib.pytest.moto_fixture(
  services=["sqs"],
  scope="class",
)

Configuring Services

Configuration of a service may be either a list of names List[str] or a list of configs List[dict] containing boto3 parameters.

Service	List of Names	List of Configs
s3	yes	yes
sqs	yes	yes
kinesis	yes	yes
dynamodb		yes
lambda		yes
sns	yes	yes

For example, your configuration might look like this:

S3 = ["first-bucket", "second-bucket"]

SQS = ["first-queue", "second-queue"]

KINESIS = ["first-stream", "second-stream"]

DYNAMODB = [
    {
        "AttributeDefinitions": [
            {"AttributeName": "uri", "AttributeType": "S"},
            {"AttributeName": "timestamp", "AttributeType": "S"},
        ],
        "TableName": "test-dbd-table",
        "KeySchema": [
            {"AttributeName": "uri", "KeyType": "HASH"},
            {"AttributeName": "timestamp", "KeyType": "RANGE"},
        ],
    }
]

LAMBDA = [
    {
        "zip_path": "dist/build.zip",
        "FunctionName": "my_lambda",
        "Runtime": "python3.6",
        "Environment": {"foo": True},
    }
]

SNS = [
    "my-topic-with-default-attrs",
    {
        "Name": "my-topic-with-additional-params",
        "Tags": [{"Key": "key1", "Value": "val1"}],
        "Attributes": {
            "DisplayName": "YourSystemIsOnFireTopic",
        },
    }
]

These configurations don't have to be static. You could use a pytest fixture to build or compile a list of resources that you want mocked.

Using both moto and localstack

You can point the aws fixture at moto or localstack to explicitly to require a specific stack to exist for the duration of your service fixture. For example, if you use both stacks:

# conftest.py
stack_config = {
    "services": ["dynamodb", "kinesis", "sqs", "s3", "lambda"],
    "scope": "class",
    "autouse": False,
    "region_name": "us-east-1",
}

localstack = pytest_localstack.patch_fixture(**stack_config)
moto = b3f.contrib.pytest.moto_fixture(**stack_config)

@pytest.fixture(scope="class")
def aws(moto):
    pass

# component/conftest.py
@pytest.fixture(scope="class")
def aws(localstack):
    pass

Known Issues

• Using both pytest-localstack and moto in the same project may break if the pytest-localstack tests run first. It's suspected this is due to an issue with cleanup with the pytest-localstack session, but is this is still under investigation.