apetest 0.1.1

Smarter-than-monkey testing for web apps

APE - Automated Page Exerciser

Smarter-than-monkey testing for web apps

https://boxingbeetle.com/tools/ape/

Testing Without Test Scripts

The only thing APE needs to know about a web app or site is its URL. From that starting point, it automatically finds links to other pages and subjects all of them to a series of checks. So it takes very little effort to check your web app or site with APE.

Of course, since APE knows next to nothing about your specific web app, there are limits to what it can check. For example, it can submit a form and check whether the server returns an error or an OK result and whether the resulting page is valid HTML, but it cannot check whether the content matches the submitted form values. Therefore, APE is not a replacement for scripted test tools like Selenium.

You may be surprised by how many problems can be uncovered just following random links and checking the result. Give APE a try!

Can APE Test My Web App/Site?

APE can be used if:

• you have a static web site, for example one written by hand or generated by a static content management system like Lektor or Jekyll
• you have a dynamic web application in which URLs are meaningful and the main content is in HTML

APE cannot be used if:

• your server relies on session state instead of HTTP GET requests to determine what to render
• your web app generates all HTML in client-side JavaScript instead of serving it over HTTP

So roughly, APE will work if your app supports deep-linking and JavaScript is used for optional enhancements, while APE will not work if you have built a single-page application.

Installation

APE requires Python 3.5 or higher.

You can use the package tool "pip" that comes with Python to install APE:

$ pip install apetest

Now you should be able to run the apetest command:

$ apetest --version
APE 0.1.0

APE uses the Nu Html Checker (v.Nu) to check HTML. The checker itself is installed by pip, but you also need a Java runtime on your machine to run it, such as OpenJDK. You only need the runtime (JRE) and not the full development kit (JDK).

You can use APE without the HTML checker, but it is much more useful with the checker enabled.

Usage

To check the HTML of a static web site in a local directory:

$ apetest --check launch website/ report.html

To do a quick sanity check (no HTML check) of a web application in development:

$ apetest localhost:8080 report.html

To see all command line options:

$ apetest --help

How Mature Is APE?

It is mature enough that I'm using it in production to test our company web site and the web application SoftFab that we'll be releasing soon. However, at the time of writing I am the only user, so I fully expect that once other people start using it, they will run into bugs and limitations that I haven't. If that happens to you, please let me know (see next section).

The code was modernized over the last few months and should be in reasonable shape. It is fully documented and free of issues PyLint can detect. It is missing unit tests for most of its functionality though, so I'm relying on system tests at the moment, which can easily miss corner cases.

Feedback

You can report bugs and make feature requests in APE's issue tracker at GitHub.

If you are interested in sponsoring a new feature, contact us at Boxing Beetle.

Writing a Plugin

APE can be extended using plugins. Plugins are Python modules placed into the apetest.plugin package; read the API documentation of apetest.plugin to learn about the plugin interface.

The plugin interface was designed to have just enough features to be able to convert pre-existing optional functionality to plugins. The intention is to grow and improve it over time. So if you need additional callbacks or information passed, feel free to request a change in the plugin interface, or prototype one yourself (see next section).

Contributing

If you want to modify APE, start by cloning the Git repository:

$ git clone https://github.com/boxingbeetle/apetest.git

Make sure the python command starts Python version 3.5 or later. If this is not the default on your system, you can use for example pyenv or virtualenv to set up an environment with the right Python version.

APE uses the Poetry build system for managing its development environment. Using the recommended installation procedure instead of pip helps separate Poetry's dependencies from those of the software it manages, like APE.

Start a shell in the virtual environment managed by Poetry:

$ poetry shell

Install APE and its runtime and development dependencies:

$ poetry install

Now you should have an apetest command available in the Poetry shell that runs APE directly from the source tree. This means that any code you edit is immediately active, for easy testing.

One of the tools Poetry will install for you is Invoke, which can be used to perform a few useful developer tasks. You can see the full list of tasks by running:

$ inv --list

The task you will probably want to run first is docs, which generates the documentation files:

$ inv docs

Now you can read the API documentation in docs/api/. The documentation of the top-level module gives a quick overview of the code.

Before submitting a pull request, please run inv test to run all tests. There should be no failing tests and zero warnings from PyLint. If PyLint detects any false positives, please add a pylint: disable=<id> comment to suppress them.

Note that APE may find invalid HTML in the API docs, but as long as APE doesn't crash and produces a readable report, that counts as the test passing as far as APE is concerned.