MUTERIA (MUlti-Tools and criTERIA framework for automated software testing)
https://github.com/muteria/muteria
Muteria is a Software Analysis/Testing framework that integrate multiple tools. Muteria support tools from various programming languages which are supported by implementing a driver.
Report Sample.
Muteria can be used through:
- Its API
- Commnad Line Interface (CLI)
Suported Systems
Muteria is written in Python and thus can run on Windows, Linux or macOS.
Installation
muteria requires Python 3.
- Install Muteria by running:
pip install muteria
- View the usage help:
muteria --help
Usage
Muteria requires to have the underlying tools installed on the system.
Usage example C (in Docker container)
A docker image, with preinstalled tools, can be used to run muteria on a sample C language program. The installed tools are: GNU GCov, KLEE, Shadow, Mart, SEMu.
- Pull the docker image:
docker pull thierrytct/cm
- run the docker image in a container:
docker run -it --rm thierrytct/cm bash
- Download the example program:
git clone https://github.com/muteria/example_c.git
- Change into the example program directory:
cd example_c
- run using the configuration file in ctrl/conf.py
muteria --config ctrl/conf.py --lang c run
Usage example Python
Example of measuring coverage for a python program using coverage.py.
- Install
coverage.py:
pip install coverage
- Download the example program:
git clone https://github.com/muteria/example_python.git
- Change into the example program directory:
cd example_python
- run using the configuration file in ctrl/conf.py
muteria --config ctrl/conf.py --lang python run
Drivers development
Testcase Tools Drivers
Details on writing drivers for testcase (test generation) tools can be found here.
Test Criteria Tools Drivers
Details on writing drivers for test criteria tools can be found here.
Current Limitation
- Interface difference between multiple versions of the same tool
Development
Things to fix
-
Ensure tools plugins do not leave the repo srcs or exes in an infected state. They must cleanup the changes made. Ensure that by cleaning for them at top level (when the driver's methods are called in the meta tools)
-
Handle reporting test error in stats (first within meta test execution)
-
Enable having no criterion set
-
Complete and test the web UI.
-
Choice of test level (unit, system, ...)
-
Complete the documentation
-
Augment the test suite
Features
- parallelism
- Implement option to run phases of the analysis in parallel.
- Implement option to run tests in parallel
- Web Interface
- Front-end UI
- server (back-end)
- Write custom configuration function on-line
- Write manual tests on-line
- Command Line Interface
- basic interaction to launch execution
- Library API interface
- API in docstring
- make the documented API available
- Reporting
- Basic metric display
- cross project reporting (for experiment and evolution)
- plotting
- Checkpointing and Logging
- checkpoint and logging implemented
- logging optional
- checkpointing optional
- Multiple projects
- obtain data from previous versions execution in current (regression)
- merge data across multiple project (for experiments)
- Test Adequacy Criteria(TAC)
- Support adding new TACs
- Support enabling/disabling TAC during execution
- Tools
- support plugging in test generation tools
- support plugging in TAC instrumenting tools
- support plugging in TAC test objectives selection/prioritization.
- support plugging in test case selection prioritization techniques.
- support adding test formats (for test execution)
- support adding build systems.
- support custom build script
- Programming languages
- support adding programming languages
- support multi-languages tools
1 maintainer
Release files for muteria 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| muteria-0.2.1.tar.gz | 344.7 kB | Details |
Release files / muteria-0.2.1.tar.gz
| Download URL | muteria-0.2.1.tar.gz |
|---|---|
| Size | 344.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6003af833489aa44657bcf41b43724fa31bcf7b2df5aad797a5c1e7216b92fd2
|
|
BLAKE2b-256 checksum How to use checksums |
820d487fa8de3f4d3e9782f8941b6797070c972281f09c4f033f3acc3c0de302
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.9.5
|
