Skip to main content

Universal regexp-based mutation tool

Project description

This is a tool based purely on regexp-specified rewrite of code lines for mutation generation, including multi-language rules aided by special rules for languages or even projects.

More information on this project can be found in our 2018 ICSE tool paper:

A guest blog post for Trail of Bits shows how to use the universalmutator to help improve a C/C++ API fuzzing effort using DeepState and libFuzzer.

The universalmutator has support for extracting coverage information to guide mutation from the TSTL testing tool for Python.


To use this, you should really just do:

pip install universalmutator


mutate --help




mutate foo.swift

should, if you have the appropriate compilers installed, generate a bunch of valid, non-trivially redundant, mutants.


Sometimes the mutated code needs to be built with a more complicated command than a simple compiler call, and of course you want help discovering which mutants are killed and not killed. For example, to mutate and test mutants for the mandelbrot plotting example included in the PROGRAMMING RUST book (, just do this:

git clone
cd mandelbrot
cargo build
target/debug/mandelbrot origmandel.png 1000x750 -1.20,0.35 -1,0.20
mkdir mutants
mutate src/ --mutantDir mutants --noCheck
analyze_mutants src/ "cargo clean; cargo build; rm mandel.png; target/debug/mandelbrot mandel.png 1000x750 -1.20,0.35 -1,0.20; diff mandel.png origmandel.png" --mutantDir mutants

(It will go faster if you edit to lower the maximum number of threads used to something like 8, not 90.) At the moment, this won't use any Trivial Compiler Equivalence, but still kills about 60% of the 1000+ mutants. The killed mutant filenames will be in killed.txt and the non-killed ones in not-killed.txt.

Working with something like maven is very similar, except you can probably edit the complicated build/clean stuff to just a 'mvn test' or similar.


The tool will likely mutate other things, if you tell it they are "c" or something, but there is auto-detection based on file ending and specific rule support for:


(the last three are smart contract languages for the Ethereum blockchain).

All but C, C++, JavaScript, and Go will try, by default, to compile the mutated file and use TCE to detect redundancy. Of course, build dependencies may frustrate this process, in which case --noCheck will turn off TCE and just dump all the mutants in the directory, for pruning using a real build process. In the long run, we plan to integrate with standard build systems to avoid this problem, and with automated test generation systems such as TSTL ( for Python or Echidna for Solidity ( Even now, however, with analyze_mutants it is fairly easy to set up automatic evaluation of your automated test generator.


The universalmutator has been most frequently applied to smart contracts written in the Solidity language. It supports a few special features that are particularly useful in this context.

First, Solidity libraries are often written with only internal functions --- and the compiler will not emit code for such functions if you compile a library by itself, resulting in no non-redundant mutants. In order to handle this case, mutate can take a --compile option that specifies another file (a contract using the library, or the tests in question) that is used to check whether mutants are redundant.

Second, swapping adjacent lines of code is a seldom-used mutation operator that is unusually attractive in a Solidity context because swapping a state-changing operation and a requirement may reveal that testing is incapable of detecting some re-entrancy vulnerabilities. The testing may notice the absence of the check, but not a mis-ordering, and these mutants may reveal that. To add code swaps to your mutations, just add --swap to the mutate call. Note that swaps work in any language; they are just particularly appealing for smart contracts.


For much more information, again see -- demo/tool paper at ICSE 18.

The aim of this project is partly to see how quickly mutation can be applied to new languages, partly how much the work of a tool can be offloaded to the compiler / test analysis tools.

Authors: Alex Groce, Josie Holmes, Darko Marinov, August Shi, Lingming Zhang

Project details

Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release. See tutorial on generating distribution archives.

Built Distributions

universalmutator-1.0.25-py3-none-any.whl (36.6 kB view hashes)

Uploaded py3

universalmutator-1.0.25-py2-none-any.whl (36.6 kB view hashes)

Uploaded py2

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page