Skip to main content

Icetea - test framework

Project description

# Command-line test framework

*Icetea* is an [mbed](www.mbed.com) test framework written
with python 2.7. It is generally used to verify the ARM mbed
IoT Device Platform provides the operating system and cloud services.

Python 3 is preliminarily supported, but there might be some
compatibility issues that will be fixed as they are found.

When testing [`mbed OS`](https://www.mbed.com/en/platform/mbed-os/)
*Icetea* allows you to execute commands remotely via
the command line interface in board (`DUT`).
The interface between the test framework and `DUT` can be
for example UART, sockets or for example stdio (process `DUT`).

A more detailed description of the *Icetea* concept is
available [here](doc/README.md).

## Installation

`> python setup.py install`

When installing Icetea, installing to a
[virtualenv](https://virtualenv.pypa.io/en/stable/installation/) is
a good idea.

### Required dependencies:

* In case you want to use **local mbed devices**
* [mbed-ls](https://github.com/armmbed/mbed-ls)
* To enumerate connected boards
* [mbed-flasher](https://github.com/ARMmbed/mbed-flasher)
* to flash devices automatically
* Both of these should be installed automatically by
Icetea installation.

**OS specific:**

Linux:
* python-dev and python-lxml
`sudo apt-get install python-dev python-lxml`
* For documentation install sphinx:
`apt-get install python-sphinx`

OS X:
* [XCode developer tools](http://osxdaily.com/2014/02/12/install-command-line-tools-mac-os-x/)
* [MacPorts](https://www.macports.org/install.php)
* lxml as described
[here](http://lxml.de/installation.html#installation):
`STATIC_DEPS=true sudo pip install lxml`
* For documentation install sphinx:
`sudo port install py27-sphinx`, `sudo port select --set python python27`,
`sudo port select --set sphinx py27-sphinx`

Windows:
* python-lxml installation is problematic on Windows since
it usually requires build tools. It can however be installed
from pre-built binaries.
* Download binary for you system from the internet.
* Navigate to the directory where you downloaded the
binary and install it with `pip install <insert_file_name>`
* For documentation install sphinx with pip:
`pip install sphinx`

### Optional dependencies

* If you wish to decorate your console log with all kinds of colors,
install the coloredlogs module using pip. `pip install coloredlogs`
* There have been issues with coloredlogs installation on Windows.
We might switch to a different module at some point to enable
colored logging on Windows as well.
* [valgrind](http://valgrind.org)
* [gdb](https://www.gnu.org/software/gdb/)


### Installation step-by-step

```
git clone https://github.com/ARMmbed/icetea.git
cd icetea
python setup.py install
```

**Linux**

In order to run test cases with hardware in Linux without sudo rights:

```
sudo usermod -a -G dialout username
Log out & log in back to Linux
```

This command will add the user 'username' to the 'dialout' group and
grant the required permissions to the USB ports.

## Usage

To print the help page:

`icetea --help`

All cli parameters are described in [Icetea.md](doc/Icetea.md)

To list all local testcases from the `./testcases` subfolder:

`icetea --list`

## Test Case API

To execute a single command line:

` execute_command(<dut>, <cmd>, (<arguments>) ) `

or alias `command(...)`


```
dut[number] #dut index (1..)
dut[string] #dut nick name, or '*' to execute command in all duts
cmd[string] #command to be executed
arguments[dictionary] #optional argument list
wait = <boolean> # whether retcode is expected before continue next command. True (default) or False
expected_retcode = <int> # expected return code (default=0)
timeout=<int> # timeout, if no retcode receive
```

## Documentation
HTML documentation for Icetea can be built using sphinx. The source
for the documentation is located in [doc-source](doc-source).
For installation of sphinx see [installation](#installation).

A build script for the documentation is included in build_docs.py.
It's a simple python scripts that also generates autodoc api documentation.
Run the script with:

`python build_docs.py`

Currently similar documentation is available in markdown format in
[doc](doc). This documentation will be phased out and is not maintained
as well as the sphinx documentation.

## Examples

**Note:** Following examples uses [`dummyDut`](test/dut/dummyDut.c)
application. It works in unix systems.
To run following examples please compile `dummyDut` first using make:
`> make -C test/dut`.

To print available parameters:

`> icetea --help`

To run a single test case with the process dut:

`> icetea --tc test_cmdline --tcdir examples --type process --bin ./test/dut/dummyDut`

To run all existing test cases from the `examples` folder:

`> icetea --tc all --tcdir examples --type process --bin ./test/dut/dummyDut`

To debug dut locally with [GDB](https://www.gnu.org/software/gdb/):

**Note:** You have to install [gdb](https://www.gnu.org/software/gdb/) first (`apt-get install gdb`)

```
> icetea --tc test_cmdline --tcdir examples --type process --gdb 1 --bin ./test/dut/dummyDut
> sudo gdb ./CliNode 3460
```

To debug dut remotely with GDB server:

```
> icetea --tc test_cmdline --tcdir examples --type process --gdbs 1 --bin ./test/dut/dummyDut
> gdb ./test/dut/dummyDut --eval-command="target remote localhost:2345"
```

To analyse memory leaks with valgrind:

**Note:** You have to install [valgrind](http://valgrind.org) first (`apt-get install valgrind`)
```
> icetea --tc test_cmdline --tcdir examples --type process --valgrind --valgrind_tool memcheck --bin ./test/dut/dummyDut
```

## Running unit tests with *Icetea*

To build a test application for DUT and execute the test:

```
> make
> coverage run -m unittest discover -s test
```

To generate a coverage report (excluding plugins):

```
> coverage html --include "icetea_lib/*" --omit "icetea_lib/Plugin/plugins/*"
```

To run unit tests for plugins that ship with Icetea:

```
> coverage run -m unittest discover -s icetea_lib/Plugin/plugins/plugin_tests
```

To generate a coverage reports for plugin unit tests run:

```
> coverage html --include "icetea_lib/Plugin/plugins/*" --omit "icetea_lib/Plugin/plugins/plugin_tests/*"
```

### Dependencies

Unit tests depend on mock, coverage and netifaces.

```
> pip install mock
> pip install netifaces
> pip install coverage
```

## License

See the [license](LICENSE) agreement.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
icetea-1.0.0rc3.tar.gz (148.6 kB) Copy SHA256 hash SHA256 Source None Aug 14, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page