Crashbang 0.0.2

A tool to aid in testing the stability of programs

crashbang

A tool to aid in testing the stability of programs

Crashbang is a tool that allows you to easily run a program through several iterations, speeding up the process of testing a program's stability. It also gathers data about the iterations and can provide statistics about successful exits versus crashes. Data is presented upon test run completion and program output can be saved to a file for later analysis. There are also facilities for limited automated testing.

Crashbang is essentially a beefed up loop. It runs a program through the specified number of iterations and detects what status the program exited with. Non-zero exit status constitutes an error, so those are flagged as failures, while zero is counted as a success. If a timeout is specified and the program doesn't exit before the timeout is reached, crashbang will automatically close the program and the iteration is counted as a success. Crashbang can also detect crashed caused by a signal (e.g. SIGIOT, Segmentation faults, etc.) and will display the recieved signal in the output file.

Usage

Basic usage is to prepend crashbang to the program you want to run. This will run through 10 iterations of the program, waiting for the program to naturally terminate before proceeding to the next one.

crashbang program-name

Adding Arguments

Arguments to pass to the tested program are specified using the -a flag, and must be wrapped in quotation marks.

crashbang program-name -a "--first-argument --second-argument=true"

Options

Iterations

Crashbang supports any number of iterations. Iterations can be set using the -i flag. The default number of iterations is 10.

crashbang program-name -i 100

Timeout

A timeout can also be specified. This is helpful when creating large automated testing runs. With the timeout specified, Crashbang waits for natural program termination up to the specified timeout. If the timeout is reached, the program is stopped and the iteration is counted as a success, then a new iteration is started. If the program exits with 0 prior to the timeout, the iteration is also considered successful. Crashes before the timeout are still treated as failures, and if the program exits for any reason before the timeout is reached, the remainder of the timeout is skipped and the next iteration begins immediately.

Timeouts can be specified in seconds with the '-t' flag:

crashbang program-name -t 10

Output

Upon completion or termination of a test run, Crashbang will display the gathered statistics from the test run, including the program name, the total number of iterations, the number of successful iterations, the number of crashes detected, and the failure percentage.

Crashbang currently handles the KeyboardInterrupt exception to allow it to display statistics for completed iterations if the user uses Ctrl-C to exit the test run early. The output will reflect the total number of completed iterations in addition to the successes and crashes.

Output File

Crashbang also outputs a file with the console output (stdout and stderr) of any failed iterations. It also outputs the iteration number and the exit status ( or signal, if the termination was caused by a signal).

Planned Features

The following features are currently not implemented, but are planned for inclusion:

• Performance/timing information
• Status and ETA for completion of a test run
• Expanded output control
• Hooks for other automated testing tools