Skip to main content

Meta-tool to test a contract with various configs, using Echidna processes ("parade" is the proper name for a group of echidna)

Project description

This is an experimental script that uses configuration variance and a common corpus to try to throughly test a smart contract (or multiple contracts) with the Echidna smart-contract fuzzer (https://github.com/crytic/echidna).

It runs Echidna instances in parallel, and uses two basic ideas derived from my research:

In particular, after an initial run to generate low-hanging-fruit easy pickings in terms of coverage, future runs will consist of a number of parallel runs (determined by an --ncores argument or however many cores Python thinks you have!) where each run randomly blacklists some functions (if you have a blacklist already, it's respected and added to), and the sequence length varies, by default between 10 and 300, with a bias towards using the default sequence length of 100 or whatever length you specified.

Usage is almost like Echidna; in fact if you just use the same arguments as to Echidna, it'll probably work. E.g.,

> echidna-parade contract.sol --config config.yaml --contract TEST

will likely do something reasonable. By default "generations" of testing are 5 minutes in length, and the testing runs for an hour.


Try out the example in the 'examples' directory. Compare:

> echidna-test justlen.sol --config config.yaml --contract TEST

vs. what you can get with some knowledge of which functions not to omit from tests, and the same 120 seconds of testing with fast swarm generations:

echidna-parade justlen.sol --config config.yaml --contract TEST --timeout 120 --gen_time 30 --ncores 8 --always "TEST.turn_on_length_checking()" "TEST.push_1()" "TEST.plus5()" "TEST.test_long_64()" "TEST.test_long_128()"
Starting echidna-parade with config=Config(files=['/Users/adg326/echidna-parade/examples/justlen.sol'], name='parade.82538', contract='TEST', config=<_io.TextIOWrapper name='config.yaml' mode='r' encoding='UTF-8'>, ncores=8, corpus_dir=None, timeout=120, gen_time=30, seed=None, minseqLen=10, maxseqLen=300, prob=0.5, always=['turn_on_length_checking', 'push_1', 'plus5', 'test_long_64', 'test_long_128'])

Results will be written to: /Users/adg326/echidna-parade/examples/parade.82538
Identified 14 public functions: push_1, pop_1, double, plus5, halve, decimate, empty1, empty2, empty3, turn_on_length_checking, turn_off_length_checking, test_long_8, test_long_64, test_long_128

RUNNING INITIAL CORPUS GENERATION
- LAUNCHING echidna-test in parade.82538/initial blacklisting [  ] with seqLen 100
parade.82538/initial FAILED
NEW FAILURE: assertion in test_long_8: failed!💥  

SWARM GENERATION #1: ELAPSED TIME 38.62 SECONDS / 120
- LAUNCHING echidna-test in parade.82538/gen.1.0 blacklisting [ halve, decimate, empty3 ] with seqLen 100
- LAUNCHING echidna-test in parade.82538/gen.1.1 blacklisting [ pop_1, halve, decimate, empty2, empty3, test_long_8 ] with seqLen 100
- LAUNCHING echidna-test in parade.82538/gen.1.2 blacklisting [ pop_1, double, halve, decimate, empty1, empty2, empty3 ] with seqLen 239
- LAUNCHING echidna-test in parade.82538/gen.1.3 blacklisting [ empty1, empty2, empty3, test_long_8 ] with seqLen 296
- LAUNCHING echidna-test in parade.82538/gen.1.4 blacklisting [ double ] with seqLen 100
- LAUNCHING echidna-test in parade.82538/gen.1.5 blacklisting [ empty1, turn_off_length_checking, test_long_8 ] with seqLen 208
- LAUNCHING echidna-test in parade.82538/gen.1.6 blacklisting [ pop_1, decimate, empty2 ] with seqLen 100
- LAUNCHING echidna-test in parade.82538/gen.1.7 blacklisting [ pop_1, double, halve, turn_off_length_checking, test_long_8 ] with seqLen 86
parade.82538/gen.1.2 FAILED
NEW FAILURE: assertion in test_long_64: failed!💥  
NEW FAILURE: assertion in test_long_128: failed!💥  
parade.82538/gen.1.0 FAILED
parade.82538/gen.1.1 FAILED
parade.82538/gen.1.4 FAILED
parade.82538/gen.1.6 FAILED

SWARM GENERATION #2: ELAPSED TIME 86.73 SECONDS / 120
- LAUNCHING echidna-test in parade.82538/gen.2.0 blacklisting [ decimate, empty1, empty2, empty3 ] with seqLen 19
- LAUNCHING echidna-test in parade.82538/gen.2.1 blacklisting [ pop_1, empty1, empty3, turn_off_length_checking, test_long_8 ] with seqLen 103
- LAUNCHING echidna-test in parade.82538/gen.2.2 blacklisting [ halve, empty1, empty2, empty3, test_long_8 ] with seqLen 226
- LAUNCHING echidna-test in parade.82538/gen.2.3 blacklisting [ halve, decimate, empty1, empty3 ] with seqLen 100
- LAUNCHING echidna-test in parade.82538/gen.2.4 blacklisting [ pop_1, double, decimate, empty2, test_long_8 ] with seqLen 100
- LAUNCHING echidna-test in parade.82538/gen.2.5 blacklisting [ double, empty1, turn_off_length_checking, test_long_8 ] with seqLen 20
- LAUNCHING echidna-test in parade.82538/gen.2.6 blacklisting [ pop_1, halve, empty2, turn_off_length_checking, test_long_8 ] with seqLen 194
- LAUNCHING echidna-test in parade.82538/gen.2.7 blacklisting [ halve, decimate, empty1, empty2, empty3, turn_off_length_checking ] with seqLen 100
parade.82538/gen.2.4 FAILED
parade.82538/gen.2.3 FAILED
parade.82538/gen.2.7 FAILED
parade.82538/gen.2.0 FAILED
parade.82538/gen.2.1 FAILED
parade.82538/gen.2.5 FAILED
parade.82538/gen.2.2 FAILED
parade.82538/gen.2.6 FAILED
DONE!

SOME TESTS FAILED

Property results:
========================================
assertion in test_long_8: failed!💥  
FAILED 8 TIMES
See: parade.82538/initial/echidna.out, parade.82538/gen.1.2/echidna.out, parade.82538/gen.1.0/echidna.out, parade.82538/gen.1.4/echidna.out, parade.82538/gen.1.6/echidna.out, parade.82538/gen.2.3/echidna.out, parade.82538/gen.2.7/echidna.out, parade.82538/gen.2.0/echidna.out
========================================
assertion in test_long_64: failed!💥  
FAILED 9 TIMES
See: parade.82538/gen.1.2/echidna.out, parade.82538/gen.2.4/echidna.out, parade.82538/gen.2.3/echidna.out, parade.82538/gen.2.7/echidna.out, parade.82538/gen.2.0/echidna.out, parade.82538/gen.2.1/echidna.out, parade.82538/gen.2.5/echidna.out, parade.82538/gen.2.2/echidna.out, parade.82538/gen.2.6/echidna.out
========================================
assertion in test_long_128: failed!💥  
FAILED 11 TIMES
See: parade.82538/gen.1.2/echidna.out, parade.82538/gen.1.0/echidna.out, parade.82538/gen.1.1/echidna.out, parade.82538/gen.2.4/echidna.out, parade.82538/gen.2.3/echidna.out, parade.82538/gen.2.7/echidna.out, parade.82538/gen.2.0/echidna.out, parade.82538/gen.2.1/echidna.out, parade.82538/gen.2.5/echidna.out, parade.82538/gen.2.2/echidna.out, parade.82538/gen.2.6/echidna.out

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for echidna-parade, version 0.1.1
Filename, size File type Python version Upload date Hashes
Filename, size echidna_parade-0.1.1-py2-none-any.whl (7.6 kB) File type Wheel Python version py2 Upload date Hashes View
Filename, size echidna_parade-0.1.1-py3-none-any.whl (7.6 kB) File type Wheel Python version py3 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page