CobraFuzz: Parallel coverage-guided fuzz testing for Python

CobraFuzz is a parallel coverage-guided fuzzer for testing Python programs. It is a fork of pythonfuzz.

Fuzzing for safe languages like Python is a powerful strategy for finding bugs like unhandled exceptions, logic bugs and security bugs. It makes a good addition to classic unit-testing.

Usage

Fuzz Target

The first step is to implement a function to be fuzz-tested, also called a fuzz target. Here is an example of a simple fuzz target for the built-in html module

from cobrafuzz.main import CobraFuzz


@CobraFuzz
def fuzz(buf: bytes) -> None:
    from html.parser import HTMLParser

    try:
        parser = HTMLParser()
        parser.feed(buf.decode("ascii"))
    except UnicodeDecodeError:
        pass


if __name__ == "__main__":
    fuzz()

Features of a fuzz target:

• fuzz() will call the fuzz target in an infinite loop with random data generated by the coverage-guided algorithm. The data is passed to the target in a separate process through the buf parameter.
• The function must catch and ignore any expected exceptions that arise when passing invalid input to the tested program.
• The fuzz target must call the test function / library with the passed buffer or a transformation on the test buffer if the structure is different or of different type.
• Fuzz functions can also implement application level checks to catch application or logical bugs. For example: decode the buffer with the library under test, encode it again, and check that both results are equal. To communicate the result, the fuzz target should throw an exception.
• CobraFuzz will report any unhandled exceptions as crashes.

Running

The next step is to download CobraFuzz and then run the fuzzer:

$ pip install cobrafuzz
[...]

$ python examples/fuzz_htmlparser/fuzz.py --crash-dir results fuzz --max-crashes 1
#0 READ units: 1 workers: 7 seeds: 1
#1 NEW     cov: 279 corp: 1 exec/s: 89 crashes: 0
#5 NEW     cov: 366 corp: 2 exec/s: 299 crashes: 0
#7 NEW     cov: 401 corp: 3 exec/s: 3124 crashes: 0
[...]
#154382 NEW     cov: 1086 corp: 50 exec/s: 22835 crashes: 0
#156521 NEW     cov: 1092 corp: 51 exec/s: 2797 crashes: 0
Crash dir created (results)
sample was written to results/crash-c76ab601df7d8513560ad3c1a38f43c715122b342637a4517a32d13dea03d3ce
sample = 3c215b2119
Found 1 crashes, stopping.

By default, CobraFuzz will use all CPUs available in the system (8 in this example: 1 coordination process and 7 independent workers). Use the -j / --num-workers command line parameter to override the default. In this example an unhandled exception is found in a few seconds.

Corpus

CobraFuzz will generate and test various inputs in an infinite loop.

Seeds can be provided as a mix of files and directories by the seeds parameter. If one of those seeds parameters is a directory, all regular files therein are used as seeds. CobraFuzz can also start with an empty seed corpus, though some valid test-cases in the seed corpus may speed up the fuzzing substantially.

More fuzz target examples (for real and popular libraries) can be found in the examples directory.

Credits & Acknowledgments

CobraFuzz is a fork of pythonfuzz. pythonfuzz is a port of fuzzitdev/jsfuzz. jsfuzz is in turn heavily based on go-fuzz originally developed by Dmitry Vyukov. go-fuzz is in turn heavily based on Michal Zalewski's AFL.

Contributions

Contributions are welcome, feel free to open issues and pull requests on GitHub.

Trophies

• python built-in HTMLParser - unhandled exception (pythonfuzz), twice
• beautifulsoup (pythonfuzz)

Feel free to add bugs that you found with CobraFuzz to this list via pull requests.