Skip to main content

Mutation Testing for Ethereum Smart Contracts

Project description


Build Status Gitter

Vertigo is a mutation testing framework designed to work specifically for smart contracts. This mutation testing framework implements a range of mutation operators that are either selected from previous works or tailored to solidity.

Quick Start Guide

To install vertigo, execute the following command:

pip3 install --user eth-vertigo

You can now run vertigo on a truffle project with the following command (assuming you have a development network configured in yourtruffle-config.js):

vertigo run --network development

Depending on your environment it might be required to specify the location of the truffle executable:

vertigo run --network development --truffle-location <node_dir>/bin/truffle 

There are a few additional parameters available that allow you to tweak the execution of vertigo:

$ python run --help
Usage: run [OPTIONS]

  Performs a core test campaign

  --output TEXT            Output core test results to file
  --network TEXT           Network names that vertigo can use
  --rules TEXT             Universal Mutator style rules to use in mutation
  --truffle-location TEXT  Location of truffle cli
  --sample-ratio FLOAT     If this option is set. Vertigo will apply the
                           sample filter with the given ratio
  --exclude TEXT           Vertigo won't mutate files in these directories
  --help                   Show this message and exit.

Known Issues

Ganache is generally used only for a single run of the entire test suite. For the general use case, it does not matter if Ganache creates a few thousand files. Unfortunately, once you start executing the entire test suite hundreds of times, you can end up with millions of files, and your machine could run out of free inode's. You can check whether this happens to you by running:

df -i

This issue (#1) is known, and we're working on a fix.

In the meanwhile. If your test suite is large enough to munch all your inodes, then there are two options:

  • You can use the command line option --sample-ratio to select a random subsample of the mutations (reducing the number of times that the test suite is run)
  • You can create a partition that has a sufficient amount of inodes available

Publications and Articles

Practical Mutation Testing for Smart Contracts - Joran J. Honig, Maarten H. Everts, Marieke Huisman

Introduction into Mutation Testing - Joran Honig

Mutation Testing for Smart Contracts - A step by step guide - Joran Honig

If you want to cite vertigo, please use the following:

author="Honig, Joran J.
and Everts, Maarten H.
and Huisman, Marieke",
title="Practical Mutation Testing for Smart Contracts",
booktitle="Data Privacy Management, Cryptocurrencies and Blockchain Technology",
publisher="Springer International Publishing",

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

eth-vertigo-1.3.0.tar.gz (20.7 kB view hashes)

Uploaded source

Built Distribution

eth_vertigo-1.3.0-py3-none-any.whl (40.4 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page