simple-iotest 3.66.0

A dead simple iotest for HPC environments

iotest

Ever feel like fio is too complicated and too optimized? Well, simple-iotest covers this for you!

Using the most simple non-optimized write() or writev() with fallback to python f.write() (just like your code!) to test file system io performance.

Warning: if using big file sizes, make sure you have enough memory to hold all these bits!

Note: Recommand to also install the package Tee_Logger to also log your test results to your /var/log/ ( configurable ) for future keeping.

Generated:

This script tests I/O performance by creating, reading, moving, and indexing files in various modes. It allows you to measure different aspects of disk performance and log the results.

Installation

pipx install simple-iotest

Usage

Run the script with:

iotest [options] [modes]

Default mode is write (w) when no mode is specified.

Common Options

• -fs, --file_size: Size of the test files (default: 30 MiB; suffix m, g, etc. supported).
• -fc, --file_count: Number of files to process per worker (default: 50).
• -pc, --process_count: Number of worker processes (default: 36).
• -wtpt, --write_target_throughput: Target sync write throughput (e.g. 500m, 2g); auto-tunes the first write pass.
• -rtpt, --read_target_throughput: Target sync read throughput; auto-tunes the first read pass. For simultaneous mode with both targets, the initial writer/reader split is estimated from the target ratio before tuning.
• -d, --directory: Directory for file operations.
• -q, --quiet: Suppresses output.
• -z, --zeros: Uses zero-filled data instead of random.
• -nl, --no_log: Disables log file creation.
• -nr, --no_report: Disables result report creation.

Modes

• write / w: Only file writes (default).
• read / r: Only file reads.
• index / i: Create and remove temporary index folders.
• random: Random read/write steps.
• comprehensive / c: write → move → stat → read per file.
• rw / wr: Run write, then read sequentially in batch mode.
• simultaneous / s: Pre-writes read targets, then runs half the workers writing and half reading concurrently on separate files. Removes all temp files when done.
• rwi / wri: Run write, then read, then index sequentially in batch mode.

Example:

iotest -fs 50m -fc 100 -pc 4 -d /tmp/iotest write read

Will launch 4 processes each write same random 50MiB size data to 100 seperate files sequentially. Then will launch another 4 processes reading the same files.

Simultaneous read/write example:

iotest -fs 50m -fc 20 -pc 8 -d /tmp/iotest simultaneous

Check the available arguments with -h or --help for more details.

$ iotest -h
usage: iotest [-h] [-fs FILE_SIZE] [-fc FILE_COUNT] [-t PROCESS_COUNT] [-d DIRECTORY] [-ld LOG_DIRECTORY] [-q] [-v] [-S]
              [-nl] [-nr] [-z] [-addr MESSAGE_END_POINT_ADDRESS] [--threshold_to_report_anomaly THRESHOLD_TO_REPORT_ANOMALY]
              [-V]
              [{comprehensive,read,write,random,index,r,w,rw,wr,i,rwi,wri,c} ...]

Test total disk bandwidth. Default mode is write (w). Comprehensive mode: write -> move -> stat -> read

positional arguments:
  {comprehensive,read,write,random,index,r,w,rw,wr,i,rwi,wri,c}
                        The mode the script will operate in (default: write / w). COMPREHENSIVE: per-thread write -> move -> stat -> read.
                        WRITE: batched all-thread write. READ: batched all-thread read. INDEX: creates --file_count index folders,
                        stat each, then delete. RWI / WRI: run write, then read, then index sequentially in batch mode.

options:
  -h, --help            show this help message and exit
  -fs, --file_size FILE_SIZE
                        File size (default:30), defaults to mb, can specify in t(b),g(b),m(b),k(b),b
  -fc, --file_count FILE_COUNT
                        Number of files to create and read per process (default:50)
  -t, -pc, --process_count PROCESS_COUNT
                        Number of processes to run concurrently (default:36)
  -d, --directory DIRECTORY
                        Directory to put the files in (default:<pwd>)
  -ld, --log_directory LOG_DIRECTORY
                        Directory to put the log files in (default:/var/log/)
  -q, --quiet           Suppress output, default True in new version
  -v, --verbose         Verbose output
  -S, --stealth         Suppress verbose output and verbose log file
  -nl, --no_log         Do not write log files
  -nr, --no_report      Do not write report files
  -z, --zeros           Use zeros instead of random numbers. Use this if you are sure no write compression is available. Potentially
                        higher write accuracy.
  -addr, --message_end_point_address MESSAGE_END_POINT_ADDRESS
                        The end point address of the message
  --threshold_to_report_anomaly THRESHOLD_TO_REPORT_ANOMALY
                        The threshold to report if 1 percent high is higher then 1 percent low * <threshold_to_report_anomaly>
  -V, --version         show program's version number and exit