iotsploit-fuzzer

Standalone fuzzer library extracted from zeekr_sat_main.

• Package name (PyPI): iotsploit-fuzzer
• Import name (Python): iotsploit_fuzzer

Install (editable)

pip install -e iotsploit-fuzzer

Notes

This package is being introduced to replace (and later deprecate) the in-repo iot_protocol_fuzzer module.

Two things this package fuzzes

They are not the same job, and conflating them is how a fuzzing effort ends up measuring the wrong thing.

Outbound — the device under test. CAN/UART/SPI payloads sent through a wire harness at whatever is on the other end. Needs the rig. This is what CANHarness, UARTHarness and SPIHarness do, driven from the Django UI.

Inbound — our own parsers. The same Orchestrator, pointed at a function instead of an interface. Needs nothing but CPU, so it runs in CI and in the commit gate. That is what follows.

The parser loop

A campaign that is run once, reported and forgotten finds its last bug in week two. What keeps producing information is the error boundary -- the line between the inputs a parser accepts and the ones it rejects -- because every refactor moves it and almost every move is unintended. Nothing in the test suite asserts which ARXML files import or which ASC lines parse. The ledger is that assertion, and it writes itself.

# What is in the registry
poetry run python -m iotsploit_fuzzer.core.parser_campaign --list

# One target, one campaign
poetry run python -m iotsploit_fuzzer.core.parser_campaign \
    --target canbus.scan_log --iterations 2000

# Everything, as a nightly run. Exits non-zero on a violation.
poetry run python -m iotsploit_fuzzer.core.parser_campaign --iterations 2000

# Replay the retained corpus and generate nothing. This is what the commit
# gate runs, via tests/test_parser_corpus_replay.py.
poetry run python -m iotsploit_fuzzer.core.parser_campaign --replay

A campaign reports exactly three things:

Event	Means	Nightly
VIOLATION	A contract broke: an undeclared exception, a hang, a resource limit, a broken round trip	Fails
BOUNDARY_MOVED	A payload the ledger knows now does something else. Not necessarily a bug -- a behaviour change somebody should confirm was intended	Reports
NEW_REGION	A signature this target has never produced. The corpus grew	Informational

Each parse runs in a process we are willing to lose

ParserHarness is a controller; the parse happens in a subprocess under wall-clock, memory, output-size and payload-size limits. This is not defensiveness. A parser that allocates until the host swaps cannot be recovered from in-process -- except MemoryError runs only after the allocation happened -- and a signal handler only runs at a bytecode boundary the interpreter may never reach again. Worker death is the result.

Workers are batched and recycled rather than forked per payload, because a spawn costs more than the parsing does. Anything that is not a plain accept or reject is re-run alone in a fresh worker, three times, before it is believed: an outcome that does not reproduce is reported as flaky and never enters the corpus.

The ledger is a contract, and it is versioned

corpus/<target>/ holds two files: payloads.zip and ledger.json. One archive per target rather than one file per payload -- a thousand inputs of a hundred bytes each, stored loose, was 74% of the repository's tracked file count for 1% of its bytes, and a 4 KB block apiece turned 1.1 MB into 9.4 MB on disk. An existing loose corpus migrates itself on the next save.

The ledger is tracked in git on purpose -- it is the loop's memory, the gate's regression corpus, and a boundary movement arrives as a JSON diff in the pull request that caused it. Each entry carries the payload's content hash, its normalised outcome signature, and the campaign that first saw it.

Signatures never contain a raw exception message: messages carry offsets, hex, paths and quoted input, and comparing them would report thousands of movements nobody caused. They also never contain a line number, for the same reason.

Changing a target's declared exception set, its adapter, or the signature format changes its fingerprint, and a ledger recorded under a different fingerprint is refused rather than diffed:

poetry run python -m iotsploit_fuzzer.core.parser_campaign \
    --target canbus.scan_log --rebaseline

Re-baselining keeps the payloads -- they are the expensive part -- and drops only the claim about what they do, which the next campaign re-derives.

Two mutators

The default is eight byte-level operations on a seeded PRNG. It needs nothing installed, which is the point: a campaign has to be able to run on a machine nobody has prepared -- CI, a Pi, the Windows target.

--radamsa selects radamsa instead, which reads the shape of its input, so a mutated JSON document is usually still JSON and far more mutants survive the adapter:

git clone --depth 1 https://gitlab.com/akihe/radamsa
cd radamsa && make && make install PREFIX=$HOME/.local

poetry run python -m iotsploit_fuzzer.core.parser_campaign --radamsa --iterations 2000

Measured on this registry, fresh corpus, same seed, all targets:

	signatures	time	per second
built-in, 2,000 inputs	304	41 s	7.4
radamsa, 2,000 inputs	331	848 s	0.4
built-in, 32,000 inputs	443	614 s	0.7

Per input radamsa wins by 9%; per second the built-in wins by 19x. Given equal wall clock the built-in found 34% more signatures and two defects radamsa did not -- and five of the six product defects the loop has found came from it.

Keep radamsa for what it reaches rather than for volume: it builds the deep nesting and long repetitions a byte mutator hits only by accident, which is how the RecursionError in the frame composer was found.

Both modes are seeded and reproducible, both record which payload a mutant came from, and the manifest says which one ran (builtin/<seed> or radamsa/<seed>). Neither the commit gate nor --replay uses a mutator at all.

The nightly run

tools/testing/nightly-parser-fuzz.sh runs one campaign against every target and exits non-zero on a violation. Driven by cron rather than by the platform, so that a night when Django or Redis is down is still a night the loop runs:

17 3 * * *  /path/to/repo/tools/testing/nightly-parser-fuzz.sh

The seed is the day of the year, so each night explores a different corner and any night can be reproduced exactly. Logs land in artifacts/parser-fuzz-logs/, which is git-ignored; the corpus it grows is not, and committing that change is what carries the night's learning to everyone else and puts it in the commit gate.

--iterations is the mutation budget. The retained corpus is replayed on top of it rather than out of it -- a boundary movement is defined on a payload the ledger already holds, so a corpus larger than the budget would otherwise stop the campaign mutating at all.

Triage, when it fails: the log names the target, the payload hash and the source line. corpus/<target>/payloads/<hash>.bin is the input. Fix the owner, then --replay that target to confirm; the payload stays in the corpus, so every commit from then on checks it.

Fuzzing one function

tools/testing/fuzz mypkg.parser:parse_config --raises ValueError --seed '{"port": 80}'
tools/testing/fuzz ./newfile.py:parse_range --raises ValueError --seed 'bytes=0-1023'
tools/testing/fuzz mypkg.log:scan --raises LogError --seed @capture.asc

That is the whole thing. It reads the function's signature to work out whether to hand it bytes, text, JSON or a path; it finds ValueError in builtins and LogError beside the function itself; and the corpus goes to a temporary directory unless you pass --keep DIR.

--raises is the experiment. It is the contract you are holding the function to -- the exceptions it says it can raise. Anything else escaping is the finding. Empty means "this never raises", which is right for a decoder that returns a failure object and wrong for a validator.

Get it wrong and the run stops before it starts:

Stopping: the seeds already break the contract you gave.
Every one of them raised something --raises does not cover:

    AttributeError

... Re-run with:

    --raises AttributeError

Without that check a wrong contract does not fail, it just never finishes: every input becomes a violation and every violation is replayed three times in a fresh process to confirm it.

Seeds decide how deep it gets. One real input is worth more than any number of iterations. On the same function, same budget:

Seed	Result
none	1 signature, corpus 3 -- never got past the first check
bytes=0-1023	16 signatures, corpus 36

Reading the result. violations is a broken contract, and the payload is in the corpus directory. moved is a payload that used to do something else -- not necessarily a bug. new is behaviour never seen before, and should fall towards zero as the corpus fills. Zero violations is the normal outcome; the signature count is the map of what your function does.

Fuzzing another application

The engine knows nothing about IoTSploit. Targets live in a pack -- an ordinary module that calls register() -- and IoTSploit's is just the one that ships here:

# myapp_fuzz.py, anywhere on PYTHONPATH
from iotsploit_fuzzer.harnesses.parser_targets import ParseTarget, json_object, register

def parse_config(payload: bytes):
    raw = json_object(payload, "a config object")
    ...

register(ParseTarget(
    name="myapp.config",
    adapter="myapp_fuzz:parse_config",
    declared=("builtins:ValueError",),
    seeds=(b'{"port": 8080}',),
))

python -m iotsploit_fuzzer.core.parser_campaign \
    --targets myapp_fuzz --root ~/myapp-corpus --iterations 3000

--targets replaces the default pack entirely, so none of IoTSploit's load. --root keeps the corpus with your own source, where the gate that replays it lives. A pack imports ParseTarget, register, and whichever adapter helpers it wants -- temp_file, json_input, json_object, text_input -- each of which raises Skip rather than letting a malformed payload look like a defect in your parser.

Everything else -- worker isolation, outcome signatures, the corpus, the ledger and its fingerprint, the boundary diff, the gate replay -- works the same whatever the pack contains.

Making it permanent

tools/testing/fuzz throws its corpus away, which is right for a question you are asking once. When a target earns a permanent slot, move it into a pack -- targets/iotsploit.py for this codebase -- as an adapter next to the others and an entry in the registry list:

ParseTarget(
    name="drivers.logic_capture",
    adapter=f"{_HERE}:logic_capture",
    declared=("builtins:ValueError", "builtins:TypeError"),
    seeds=(_CAPTURE_SEED,),
    budget_seconds=5.0,
),

tests/test_parser_targets.py then checks it on every commit: that the adapter and every declared name resolve, and that at least one seed actually reaches the target instead of being skipped. Once it has a corpus, the gate replays it.

Known limits

• --radamsa mutants inflate. Left alone they grow until they hit payload_max_bytes, and radamsa's cost scales with input size, so a long radamsa campaign gets slower as it goes.
• Mutation is byte-level by default. For the JSON-shaped targets (canbus.from_target, canbus.decode_frame) roughly 90% of mutants are not valid JSON and are skipped. Structure-aware mutation would fix it and has not been written.
• someip.sd_parse has almost no observable boundary from random bytes: it catches everything and returns a list, so nearly every input looks the same. It needs seeds that are valid SD datagrams to say anything.
• Novelty by outcome is a weak fitness signal next to coverage guidance. It plateaus. Real coverage feedback needs sys.monitoring (3.12+); on 3.10 it would cost a 10-30x slowdown.