benchmarkingcli 0.4.2

Android SoC benchmarking CLI for latency, loaded latency, and memory subsystem characterization

BenchmarkingCLI

A command-line tool for running latency benchmarks on Android devices. This tool automates the process of pushing benchmark binaries to connected Android devices, executing latency tests, and collecting performance metrics in CSV format.

Features

• Latency Testing: Measure memory latency across different buffer sizes on a specific core
• Loaded Latency Testing (v0.3+): Measure latency under concurrent bandwidth loads with configurable kernel modes and thread settings
• GPU Loaded Latency Testing (v0.4+): Measure memory latency under concurrent GPU load
• CSV Export: Results are automatically parsed and exported to timestamped CSV files
• CLI Interface: Easy-to-use command-line interface

Requirements

Host Machine

• Python: 3.8 or higher
• Android Debug Bridge (ADB): Version 1.0.40+
  • Linux: apt-get install android-tools-adb
  • macOS: brew install android-platform-tools
  • Windows: Download the SDK Platform Tools and add to PATH
  • Verify installation: adb version

Android Device

• USB debugging must be enabled on the device. To enable it:
  1. Go to Settings → About Phone
  2. Tap Build Number 7 times until you see "You are now a developer"
  3. Go to Settings → Developer Options
  4. Enable USB Debugging
• Connect the device via USB and accept the "Allow USB Debugging" prompt on the device
• Confirm the device is detected: adb devices

Installation

Install directly from PyPI:

pip install benchmarkingcli

Usage

Latency Test

Run a basic latency benchmark on a specific core:

# Run on default core (7)
benchmarkingcli latency

# Run on a specific core
benchmarkingcli latency --core 4

Parameters:

• --core: CPU core to run the latency test on (default: 7)
• --output-dir: Directory to save results (default: current directory)

Output

Results are exported as CSV files with the following columns:

• buffer_size_kb: Size of the buffer tested (in KB)
• min_ns: Minimum latency (nanoseconds)
• mean_ns: Mean latency (nanoseconds)
• max_ns: Maximum latency (nanoseconds)

Example output file: latency_results_20260427_115540.csv

Loaded Latency Test (v0.3+)

Run a latency benchmark under concurrent bandwidth load:

# Run with default settings
benchmarkingcli loadedlatency

# Run with custom configuration
benchmarkingcli loadedlatency --lat-buffer-mb 128 --bw-buffer-mb 128 --latency-core 7 --bw-cores 0,1,2,3 --kernel-mode TRIAD --ilp 4 --injection-rate desc --output-dir ./results

Parameters:

• --lat-buffer-mb: Latency buffer size in MB (default: 128)
• --bw-buffer-mb: Bandwidth buffer size in MB (default: 128)
• --latency-core: Core to run latency thread on (default: 7)
• --bw-cores: Comma-separated cores for bandwidth threads (default: 0,1,2,3,4,5)
• --kernel-mode: Kernel mode: SUM/COPY/UPDATE/TRIAD or 1/2/3/4 (default: TRIAD)
• --ilp: Unroll factor: 1/2/4/8/16/32 (default: 4)
• --injection-rate: Injection rate: asc or desc (default: asc)
• --output-dir: Directory to save results (default: current directory)

Important Update (v0.4+): The --tile-delay-sweep-order parameter has been renamed to --injection-rate. Supported values are asc and desc.

GPU Loaded Latency Test (v0.4+)

Run a latency benchmark under concurrent GPU load:

# Run with default settings
benchmarkingcli gpuloadedlatency

# Run with custom configuration
benchmarkingcli gpuloadedlatency --latency-core 7 --lat-buffer-mb 128 --gpu-buffer-mb 128 --kernel-mode TRIAD --output-dir ./results

Parameters:

• --latency-core: Core to run latency thread on (default: 7)
• --lat-buffer-mb: Latency buffer size in MB (default: 128)
• --gpu-buffer-mb: GPU workload buffer size in MB (default: 128)
• --kernel-mode: Kernel mode: SUM/COPY/UPDATE/TRIAD or 1/2/3/4 (default: TRIAD)
• --output-dir: Directory to save results (default: current directory)

Troubleshooting

"No ADB devices found"

• Check that your Android device is connected: adb devices
• Ensure USB debugging is enabled on the device
• Try: adb kill-server then adb start-server

"Latency test binary not found"

• Reinstall the package: pip install --force-reinstall benchmarkingcli

Binary push fails

• Check device storage: adb shell df /data/local/tmp
• Verify ADB permissions: adb shell ls -la /data/local/tmp