TestingBot for AWS CodePipeline and CodeBuild

Run TestingBot Selenium and Appium tests from AWS CodePipeline, see the results in CodeBuild test reports, and stop a deploy when TestingBot tests fail.

This repository has two parts. Use either one, or both:

Both use only the Python standard library. A stock CodeBuild image and the Lambda Python runtime need nothing extra.

Credentials

Store your TestingBot key and secret, from the member area, in AWS Secrets Manager as one secret, for example named testingbot:

{"key": "YOUR_KEY", "secret": "YOUR_SECRET"}

CodeBuild: tunnel and test reports

version: 0.2

env:
  secrets-manager:
    TESTINGBOT_KEY: testingbot:key
    TESTINGBOT_SECRET: testingbot:secret
  exported-variables:
    - TESTINGBOT_BUILD

phases:
  install:
    commands:
      - pip install --quiet testingbot-codepipeline
  pre_build:
    commands:
      - eval "$(testingbot-codebuild env)"
      - testingbot-codebuild tunnel start
  build:
    commands:
      - npm run test:e2e
  post_build:
    commands:
      - testingbot-codebuild tunnel stop
      - testingbot-codebuild report

reports:
  testingbot:
    files:
      - testingbot-reports/testingbot.xml
    file-format: JUNITXML

Your tests must send the TestingBot build and tunnel-identifier capabilities, read from the environment:

'tb:options': {
  build: process.env.TESTINGBOT_BUILD,
  'tunnel-identifier': process.env.TESTINGBOT_TUNNEL_IDENTIFIER
}

A full example is in examples/buildspec.yml.

Commands

Tests that never reported a pass/fail status to TestingBot appear as skipped, not failed.

The tunnel reads the key and secret from the environment, so they never appear on a command line or in the build log.

CodePipeline: gate a deploy on TestingBot results

1. Deploy the gate. From this repository, run sam deploy --guided. It asks for TestingBotSecretArn, the ARN of the secret above.

2. In the CodeBuild action that runs your tests, set the variable namespace to TestVariables and keep TESTINGBOT_BUILD under env.exported-variables in the buildspec.

3. Add an action after it: provider AWS Lambda, function GateFunctionName from the stack outputs, and user parameters:

   {"build": "#{TestVariables.TESTINGBOT_BUILD}"}
   

4. Allow the pipeline role lambda:InvokeFunction on the gate.

CloudFormation for steps 2-4 is in examples/pipeline.yml.

The gate always reports a result to CodePipeline, also on bad parameters or credentials, so the pipeline never waits for the action to time out.

Development

PYTHONPATH=src:tests python3 -m unittest discover -s tests -v

License

MIT