Skip to main content

Fuzz test Python modules with libFuzzer.

Project description

buildstatus

About

Use libFuzzer to fuzz test Python 3.6+ C extension modules.

Ideas

  • Use type annotations for less type errors in generic mutator.
  • Add support to fuzz test pure Python modules by generating C code from them using Cython.

Installation

clang 8 or later is required.

$ apt install clang
$ pip install pyfuzzer

Example Usage

Default mutator

Use the default mutator pyfuzzer.mutators.random when testing the module hello_world.

$ cd examples/hello_world
$ pyfuzzer run hello_world hello_world.c
$ clang -fprofile-instr-generate -fcoverage-mapping -g -fsanitize=fuzzer -fsanitize=signed-integer-overflow -fno-sanitize-recover=all -I/usr/local/include/python3.7m hello_world.c module.c /home/erik/workspace/pyfuzzer/pyfuzzer/pyfuzzer.c -Wl,-rpath /usr/local/lib -lpython3.7m -o pyfuzzer
$ clang -I/usr/local/include/python3.7m hello_world.c module.c /home/erik/workspace/pyfuzzer/pyfuzzer/pyfuzzer_print_corpus.c -Wl,-rpath /usr/local/lib -lpython3.7m -o pyfuzzer_print_corpus
rm -f pyfuzzer.profraw
./pyfuzzer corpus -max_total_time=1 -max_len=4096
INFO: Seed: 2903179615
INFO: Loaded 1 modules   (23 inline 8-bit counters): 23 [0x67945d, 0x679474),
INFO: Loaded 1 PC tables (23 PCs): 23 [0x465d30,0x465ea0),
INFO:        0 files found in corpus
Importing module under test... done.
Importing custom mutator... failed.
ModuleNotFoundError: No module named 'mutator'
Importing mutator 'pyfuzzer.mutators.random'... done.
Finding function 'test_one_input' in mutator... done.
INFO: A corpus is not provided, starting from an empty corpus
#2   INITED cov: 4 ft: 5 corp: 1/1b lim: 4 exec/s: 0 rss: 34Mb
     NEW_FUNC[1/1]: 0x4554e0 in m_tell /home/erik/workspace/pyfuzzer/examples/hello_world/hello_world.c:10
#51  NEW    cov: 6 ft: 8 corp: 2/5b lim: 4 exec/s: 0 rss: 36Mb L: 4/4 MS: 4 ChangeBinInt-CopyPart-InsertByte-InsertByte-
#412 NEW    cov: 10 ft: 12 corp: 3/9b lim: 6 exec/s: 0 rss: 36Mb L: 4/4 MS: 1 ChangeBinInt-
#468 NEW    cov: 11 ft: 13 corp: 4/14b lim: 6 exec/s: 0 rss: 36Mb L: 5/5 MS: 1 InsertByte-
#502 NEW    cov: 12 ft: 14 corp: 5/20b lim: 6 exec/s: 0 rss: 36Mb L: 6/6 MS: 4 ShuffleBytes-ChangeByte-ChangeBinInt-CopyPart-
#763 NEW    cov: 13 ft: 15 corp: 6/28b lim: 8 exec/s: 0 rss: 36Mb L: 8/8 MS: 1 CrossOver-
#1466        REDUCE cov: 13 ft: 15 corp: 6/27b lim: 14 exec/s: 0 rss: 36Mb L: 7/7 MS: 3 ChangeBinInt-CopyPart-EraseBytes-
#63426       DONE   cov: 13 ft: 15 corp: 6/27b lim: 625 exec/s: 31713 rss: 36Mb
Done 63426 runs in 2 second(s)
llvm-profdata merge -sparse pyfuzzer.profraw -o pyfuzzer.profdata
llvm-cov show pyfuzzer -instr-profile=pyfuzzer.profdata -ignore-filename-regex=/usr/|pyfuzzer.c|module.c
    1|       |#include <Python.h>
    2|       |
    3|       |PyDoc_STRVAR(
    4|       |    tell_doc,
    5|       |    "tell(message)\n"
    6|       |    "--\n"
    7|       |    "Arguments: (message:bytes)\n");
    8|       |
    9|       |static PyObject *m_tell(PyObject *module_p, PyObject *message_p)
   10|  18.4k|{
   11|  18.4k|    Py_ssize_t size;
   12|  18.4k|    char* buf_p;
   13|  18.4k|    int res;
   14|  18.4k|    PyObject *res_p;
   15|  18.4k|
   16|  18.4k|    res = PyBytes_AsStringAndSize(message_p, &buf_p, &size);
   17|  18.4k|
   18|  18.4k|    if (res != -1) {
   19|  15.5k|        switch (size) {
   20|  15.5k|
   21|  15.5k|        case 0:
   22|  1.50k|            res_p = PyLong_FromLong(5);
   23|  1.50k|            break;
   24|  15.5k|
   25|  15.5k|        case 1:
   26|  1.55k|            res_p = PyBool_FromLong(1);
   27|  1.55k|            break;
   28|  15.5k|
   29|  15.5k|        case 2:
   30|  1.65k|            res_p = PyBytes_FromString("Hello!");
   31|  1.65k|            break;
   32|  15.5k|
   33|  15.5k|        default:
   34|  10.8k|            res_p = PyLong_FromLong(0);
   35|  10.8k|            break;
   36|  2.88k|        }
   37|  2.88k|    } else {
   38|  2.88k|        res_p = NULL;
   39|  2.88k|    }
   40|  18.4k|
   41|  18.4k|    return (res_p);
   42|  18.4k|}
   43|       |
   44|       |static struct PyMethodDef methods[] = {
   45|       |    { "tell", m_tell, METH_O, tell_doc},
   46|       |    { NULL }
   47|       |};
   48|       |
   49|       |static PyModuleDef module = {
   50|       |    PyModuleDef_HEAD_INIT,
   51|       |    .m_name = "hello_world",
   52|       |    .m_size = -1,
   53|       |    .m_methods = methods
   54|       |};
   55|       |
   56|       |PyMODINIT_FUNC PyInit_hello_world(void)
   57|      1|{
   58|      1|    return (PyModule_Create(&module));
   59|      1|}

Print the function calls that found new code paths. This information is usually good input when writing unit tests.

$ pyfuzzer corpus_print
./pyfuzzer_print_corpus corpus/9ce9554501e0ceafdc80947723b2266c7d4465b1
Function:  tell
Arguments: [b"'"]
Result:    True
./pyfuzzer_print_corpus corpus/eba41d971b38378309f43d49643edf0cfaab5f4d
Function:  tell
Arguments: [b"'\x03"]
Result:    b'Hello!'
./pyfuzzer_print_corpus corpus/889343cd62dbff5263be0a245834550f8801df95
Function:  tell
Arguments: [b'']
Result:    5
./pyfuzzer_print_corpus corpus/e29871be07980068a513c9743dba073b122796b4
Function:  tell
Arguments: [True]
Traceback (most recent call last):
  File "/home/erik/workspace/pyfuzzer/pyfuzzer/mutators/random.py", line 94, in test_one_input_print
    res = func(*args)
TypeError: expected bytes, bool found
./pyfuzzer_print_corpus corpus/29a45f3baa3f6e1a904010761d9d82467ba0ee1e
Function:  tell
Arguments: [b'\x03\x00\x08']
Result:    0

Custom mutator

Use the custom mutator hello_world_mutator when testing the module hello_world.

Testing with a custom mutator is often more efficient than using a generic one.

$ cd examples/hello_world_custom_mutator
$ pyfuzzer run -m hello_world_mutator.py hello_world hello_world.c
...

Mutators

A Mutator uses data from libFuzzer to test a module. A mutator must implement the function test_one_input(module, data), where module is the module under test and data is the data generated by libFuzzer (as a bytes object).

A minimal mutator fuzz testing a CRC-32 algorithm could look like below. It simply calls crc_32() with data as its only argument.

import traceback
from .pyfuzzer.mutators.utils import print_function

def test_one_input(module, data):
    module.crc_32(data)

def test_one_input_print(module, data):
     print_function(module.crc_32, [data])

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pyfuzzer, version 0.7.0
Filename, size File type Python version Upload date Hashes
Filename, size pyfuzzer-0.7.0.tar.gz (9.0 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page