RESTinstance 1.0.0b35

Robot Framework test library for (RESTful) JSON APIs

Robot Framework test library for (RESTful) JSON APIs

Why?

1. RESTinstance relies on Robot Framework’s language-agnostic, natural language syntax for API tests. It is neither tied to any particular programming language nor development framework. Using RESTinstance requires little, if any, programming knowledge. It builts on long-term technologies with well established communities, such as HTTP, JSON (Schema), OpenAPI and Robot Framework.

2. It validates JSON using JSON Schema, guiding you to write API tests to base on constraints rather than on specific values (e.g. “email must be valid” vs “email is [e-mail]”). This approach reduces test maintenance when the values responded by the API are prone to change. Although values are not required, you can still test them whenever they make sense (e.g. GET response body from one endpoint, then POST some of its values to another endpoint).

3. It generates JSON Schema for requests and responses automatically, and the schema gets more accurate by your tests. The schema is a contract between different teams, or functions (backend, frontend, test developers), to agree on what kind of data the API handles. Additionally, you can mark validations to be skipped and rather use the tests to define how the API should work - then the schema also acts as a design. The schema can be further extended to an OpenAPI specification (manually for now, generating also this is planned), which RESTinstance can also test requests and responses against. This leads to very clean looking tests.

Installation

Python

On 3.x and 2.7, you can install the package from PyPi:

pip install --upgrade RESTinstance

Docker

The image has Python 3.6 and the latest Robot Framework:

docker pull asyrjasalo/restinstance
docker run --rm -ti --env HOST_UID=$(id -u) --env HOST_GID=$(id -g) \
  --env HTTP_PROXY --env HTTPS_PROXY --network host \
  --volume "$PWD/tests":/home/robot/tests \
  --volume "$PWD/results":/home/robot/results \
  asyrjasalo/restinstance tests

rfdocker

If you are already using rfdocker, just add RESTinstance to your requirements.txt and remove the commented lines in Dockerfile. It will be installed automatically the next time you run ./rfdocker.

To pass the proxy settings to the container and run it on host network:

RUN_ARGS="--env HTTP_PROXY,HTTPS_PROXY --network=host" ./rfdocker

Usage

Tip: You can run this README.rst as a test with Robot Framework.

The most common use cases for library are:

1. Testing HTTP JSON APIs, without necessarily knowing the responded values

*** Settings ***
Library         REST    https://jsonplaceholder.typicode.com
Documentation   The test data can be read from files, strings or dicts.
...             The validation keywords correspond to the JSON types.
...             They take in either a plain text path or a JSONPath.
...             Optionally type specific JSON Schema validations can be used.
...             You can optionally check for constant JSON values too.
...             Every request creates an instance. Can be output as JSON.
...             The validations are effective for the last created instance.
...             The scope of the created instances is this test suite.


*** Variables ***
${json}         { "id": 11, "name": "Gil Alexander" }
&{dict}         name=Julie Langford


*** Test Cases ***
GET an existing user
    GET         /users/1                  # this creates a new instance
    Object      response body
    Integer     response body id          1
    String      response body name        Leanne Graham   # quotes optional
    [Teardown]  Output                    # the current instance

GET existing users, JSONPath can be used
    GET         /users?_limit=5           # this is now the last instance
    Array       response body
    Integer     $[0].id                   1           # first id is 1
    String      $[0]..lat                 -37.3159    # any matching child
    Integer     $..id                     maximum=5   # multiple matches
    [Teardown]  Output  $[*].id           # all ids as a JSON array

POST with valid params to create an user
    POST        /users                    ${json}
    Integer     response status           201
    [Teardown]  Output                    # file_path=${OUTPUTDIR}/inst.json

PUT with valid params to update existing
    PUT         /users/2                  { "isCoding": true }
    Boolean     response body isCoding    true
    PUT         /users/2                  { "sleep": null }
    Null        response body sleep
    PUT         /users/2                  { "pockets": "", "money": 0.02 }
    String      response body pockets     ${EMPTY}
    Number      response body money       0.02
    Missing     response body moving

PATCH with valid params, use response further as a new payload
    &{res}=     GET   /users/3
    String      $.name                    Clementine Bauch
    PATCH       /users/4                  { "name": "${res.body['name']}" }
    String      $.name                    Clementine Bauch
    PATCH       /users/5                  ${dict}
    String      $.name                    ${dict.name}

DELETE existing successfully
    DELETE      /users/6
    Integer     response status           200    202     204
    Rest instances   ${OUTPUTDIR}/all_cases.json   # all instances so far

2. Testing API requests and responses against a JSON Schema document

   Examples for testing against JSON schema

3. Testing API requests and responses against a Swagger specification

   Examples for testing against Swagger 2.0 specification

See keyword documentation.

Development

Bug reports and feature requests are tracked in GitHub.

We do respect pull request(er)s. Please mention if you do not want to be listed below as contributors.

Library’s own tests

For simplicity, Docker is required for running the library’s own tests. No other requirements are needed.

To spin up the environment and run the tests:

./test

To run them on Python 2.7:

BUILD_ARGS="-f Dockerfile.python2" BUILD_NAME="restinstance-python2" ./test

System under test

The test API is implemented by mounterest, which in turn bases on mountebank.

In the scope of library’s tests, mounterest acts as a HTTP proxy to Typicode’s live JSON server and uses mountebank’s injections to enrich responses slightly, so that they better match to this library’s testing needs. Particularly, it allows to test the library with non-safe HTTP methods (POST, PUT, PATCH, DELETE) by mimicking their changes, instead of trying to issue them on the live server. The changes are cleared between the test runs.

Releasing

To generate keyword documentation:

./genlibdoc

To build and release Python package to PyPi:

./release_pypi

To release the Docker image to private Docker registry:

./release https://your.private.registry.com:5000/restinstance

To release the Docker image to DockerHub:

./release {{organization}}/restinstance

Credits

RESTinstance is licensed under Apache License 2.0 and was originally written by Anssi Syrjäsalo.

It was presented at (the first) RoboCon 2018.

Contributors:

• jjwong for helping with keyword documentation and examples (also check RESTinstance_starter_project)

• Przemysław “sqilz” Hendel for using and testing RESTinstance in early phase (also check RESTinstance-wrapper)

We use the following Python excellence under the hood:

• Flex, by Piper Merriam, for Swagger 2.0 validation

• GenSON, by Jon “wolverdude” Wolverton, for JSON Schema generator

• jsonpath-ng, by Tomas Aparicio and Kenneth Knowles, for handling JSONPath queries

• jsonschema, by Julian Berman, for JSON Schema draft-04 validation

• pygments, by Georg Brandl et al., for JSON syntax coloring, in console Output

• requests, by Kenneth Reitz et al., for making HTTP requests

See requirements.txt for all the direct dependencies.