RESTinstance 1.0.2

Robot Framework library for RESTful JSON APIs

Robot Framework library for RESTful JSON APIs

Advantages

1. RESTinstance relies on Robot Framework’s language-agnostic, clean and minimal syntax, for API tests. It is neither tied to any particular programming language nor development framework. Using RESTinstance requires little, if any, programming knowledge. It builts on long-term technologies with well established communities, such as HTTP, JSON (Schema), Swagger/OpenAPI and Robot Framework.

2. It validates JSON using JSON Schema, guiding you to write API tests to base on properties rather than on specific values (e.g. “email must be valid” vs “email is [e-mail]”). This approach reduces test maintenance when the values responded by the API are prone to change. Although values are not required, you can still test them whenever they make sense (e.g. GET response body from one endpoint, then POST some of its values to another endpoint and verify the results).

3. It generates JSON Schema for requests and responses automatically, and the schema gets more accurate by your tests. Output the schema to a file and reuse it as expectations to test the other methods, as most of them respond similarly with only minor differences. Or extend the schema further to a full Swagger spec (version 2.0, OpenAPI 3.0 also planned), which RESTinstance can test requests and responses against. All this leads to reusability, getting great test coverage with minimum number of keystrokes and very clean tests.

Installation

Pick the one that suits your environment best.

As a Python package

On 3.6, 3.7 and 2.7, you can install and upgrade from PyPi:

pip install --upgrade RESTinstance

This also installs Robot Framework if you do not have it already.

As a Docker image

RESTinstance Docker image contains Python 3.6.9 and the latest Robot Framework:

docker pull asyrjasalo/restinstance

Usage

There is a step-by-step tutorial in the making, best accompanied with keyword documentation.

Quick start

1. Create two new (empty) directories tests and results.

2. Create a new file atest/YOURNAME.robot with content:

*** Settings ***
Library         REST    https://jsonplaceholder.typicode.com
Documentation   Test data can be read from variables and files.
...             Both JSON and Python type systems are supported for inputs.
...             Every request creates a so-called instance. Can be `Output`.
...             Most keywords are effective only for the last instance.
...             Initial schemas are autogenerated for request and response.
...             You can make them more detailed by using assertion keywords.
...             The assertion keywords correspond to the JSON types.
...             They take in either path to the property or a JSONPath query.
...             Using (enum) values in tests optional. Only type is required.
...             All the JSON Schema validation keywords are also supported.
...             Thus, there is no need to write any own validation logic.
...             Not a long path from schemas to full Swagger/OpenAPI specs.
...             The persistence of the created instances is the test suite.
...             Use keyword `Rest instances` to output the created instances.


*** Variables ***
${json}         { "id": 11, "name": "Gil Alexander" }
&{dict}         name=Julie Langford


*** Test Cases ***
GET an existing user, notice how the schema gets more accurate
    GET         /users/1                  # this creates a new instance
    Output schema   response body
    Object      response body             # values are fully optional
    Integer     response body id          1
    String      response body name        Leanne Graham
    [Teardown]  Output schema             # note the updated response schema

GET existing users, use JSONPath for very short but powerful queries
    GET         /users?_limit=5           # further assertions are to this
    Array       response body
    Integer     $[0].id                   1           # first id is 1
    String      $[0]..lat                 -37.3159    # any matching child
    Integer     $..id                     maximum=5   # multiple matches
    [Teardown]  Output  $[*].email        # outputs all emails as an array

POST with valid params to create a new user, can be output to a file
    POST        /users                    ${json}
    Integer     response status           201
    [Teardown]  Output  response body     ${OUTPUTDIR}/new_user.demo.json

PUT with valid params to update the existing user, values matter here
    PUT         /users/2                  { "isCoding": true }
    Boolean     response body isCoding    true
    PUT         /users/2                  { "sleep": null }
    Null        response body sleep
    PUT         /users/2                  { "pockets": "", "money": 0.02 }
    String      response body pockets     ${EMPTY}
    Number      response body money       0.02
    Missing     response body moving      # fails if property moving exists

PATCH with valid params, reusing response properties as a new payload
    &{res}=     GET   /users/3
    String      $.name                    Clementine Bauch
    PATCH       /users/4                  { "name": "${res.body['name']}" }
    String      $.name                    Clementine Bauch
    PATCH       /users/5                  ${dict}
    String      $.name                    ${dict.name}

DELETE the existing successfully, save the history of all requests
    DELETE      /users/6                  # status can be any of the below
    Integer     response status           200    202     204
    Rest instances  ${OUTPUTDIR}/all.demo.json  # all the instances so far

3. Chose Python installation? Let’s go (not that language):

robot --outputdir results atest/

If you chose the Docker method instead (recall the story about red and blue pill here, if you want), this is quaranteed to work in most environments:

docker run --rm -ti --env HOST_UID=$(id -u) --env HOST_GID=$(id -g) \
  --env HTTP_PROXY --env HTTPS_PROXY --network host \
  --volume "$PWD/atest":/home/robot/atest \
  --volume "$PWD/results":/home/robot/results \
  asyrjasalo/restinstance atest/

Tip: If you prefer installing from source, pip install --editable . and verify the installation with robot README.rst

Contributing

Bug reports and feature requests are tracked in GitHub.

We do respect pull request(er)s. Please mention if you do not want to be listed below as contributors.

A CircleCI job is created automatically for your GitHub pull requests as well.

Local development

On Linux distros and on OS X, may make rules ease repetitive workflows:

$ make help
all_dev              (DEFAULT / make): test, install_e, atest
all_github           All branches/PRs: test, build, install, atest
all_prepypi          Pre to TestPyPI: build, publish_pre, install_pre, atest
all_pypi             Final to PyPI: build, publish_prod, install_prod, atest
atest                Run Robot atests for the currently installed package
black                Reformat ("blacken") all Python source code in-place
build                Build source and wheel dists, recreates .venv/release
clean                Pip uninstall, rm .venv/s, build, dist, eggs, .caches
docs                 Regenerate (library) documentation in this source tree
flake8               Run flake8 for detecting flaws via static code analysis
install              (Re)install the package from this source tree
install_e            Install the package as --editable from this source tree
install_pre          (Re)install the latest test.pypi.org (pre-)release
install_prod         Install/upgrade to the latest final release in PyPI
prospector           Runs static analysis using dodgy, mypy, pyroma and vulture
publish_pre          Publish dists to test.pypi.org - for pre, e.g. aX, bX, rcX
publish_prod         Publish dists to live PyPI - for final only, e.g. 1.0.1
pur                  Update requirements-dev's deps that have versions defined
retest               Run only failed unit tests if any, otherwise all
test                 Run unit tests, upgrades .venv/dev with requirements(-dev)
testenv              Start new testenv in docker if available, otherwise local
testenv_rm           Stop and remove the running docker testenv if any
uninstall            Uninstall the Python package, regardless of its origin

Running make runs rules test, install_e and atest at once, creates and uses virtualenv .venv/dev/ to ensure that no (user or system level) dependencies interfere with the process.

If make is not available, you can setup for development with:

python3 -m venv .venv/dev
source .venv/dev/bin/activate
pip install -r requirements-dev.txt
pip install --editable .

To recreate the keyword documentation from source (equals to make docs):

python3 -m robot.libdoc src/REST docs/index.html

Acceptance tests

The testapi/ is built on mountebank. You can monitor requests and responses at localhost:2525

To start the testenv and ran robot for acceptance tests:

make atest

If you have Docker available, then testenv is ran in Docker container which is recreated each time the above make rule is ran.

If Docker is not available, then testenv is ran using local mb which is installed and started as following (ran by the make rule, here for reference):

npx mountebank --localOnly --allowInjection --configfile testapi/apis.ejs

The tests are ran as following (ran by the make rule, here for reference):

python3 -m robot --outputdir results atest/

To run the acceptance tests from a dedicated Docker container, built and ran outside the the test API, and limit only to specific suite(s):

RUN_ARGS="--rm --network=host --env HTTP_PROXY --env HTTPS_PROXY \
  -v $PWD/atest:/home/robot/atest \
  -v $PWD/results:/home/robot/results" \
  ./docker/build_run_docker atest/output.robot

Host directories atest/ and results/ are accessed inside the container via the respective Docker volumes. Same arguments are accepted as for robot.

Host network is used to minimize divergence between different host OSes. Passing the proxy environment variables may not be required in your environment, but there should be no downside either. On OS X --network=host is required.

Docker releases

The Docker image is built by ./docker/build_run_docker which uses docker/Dockerfile.

Then, to tag this built image with two git tags, the timestamp and “latest”, and push it to a Docker image registry:

REGISTRY_USERNAME=yourname \
REGISTRY_URL=https://private.registry.com/ \
  ./docker/tag_and_push_docker

For Docker Hub, just organisation/username will do:

REGISTRY_USERNAME=yourname ./docker/tag_and_push_docker

Credits

RESTinstance is under Apache License 2.0 and was originally written by Anssi Syrjäsalo.

It was first presented at the first RoboCon, 2018.

Contributors:

• jjwong for helping with keyword documentation and examples (also check RESTinstance_starter_project)

• Przemysław “sqilz” Hendel for using and testing RESTinstance in early phase (also check RESTinstance-wrapper)

• Vinh “vinhntb” Nguyen, #52.

• Stavros “stdedos” Ntentos, #75.

We use following Python excellence under the hood:

• Flex, by Piper Merriam, for Swagger 2.0 validation

• GenSON, by Jon “wolverdude” Wolverton, for JSON Schema generator

• jsonpath-ng, by Tomas Aparicio and Kenneth Knowles, for handling JSONPath queries

• jsonschema, by Julian Berman, for JSON Schema validator

• pygments, by Georg Brandl et al., for JSON syntax coloring, in terminal Output

• requests, by Kenneth Reitz et al., for making HTTP requests

See requirements.txt for all the direct run time dependencies.

REST your mind, OSS got your back.