easybench 0.1.29

A simple and easy-to-use Python benchmarking library

EasyBench

Docs

A simple and easy-to-use Python benchmarking library.

Features

• Three flexible benchmarking styles (decorator, class-based, and command-line)
• Measure both execution time and estimated memory usage (see limitations)
• Visualization of benchmark results as boxplots for analyzing distribution and outliers
• Parametrized benchmarks to compare the same function with different input sizes
• A pytest-like fixture system for easy test data setup
• Complete lifecycle hooks (setup/teardown) for fine-grained benchmark control
• Switch between normal function execution and benchmarked execution on demand
• Customizable benchmark configuration with sorting and formatting options
• Command-line tool to run multiple benchmarks at once
• Multiple output formats (text tables, CSV, JSON, pandas.DataFrame)
• Extensible reporting system for custom output destinations

Installation

pip install easybench

Optional Dependencies

EasyBench supports optional dependencies for additional features:

# Install with visualization support
pip install easybench[all]

The all option includes:

• matplotlib: For visualization and plotting benchmark results
• seaborn: For enhanced statistical visualizations
• pandas: For outputting benchmark results as DataFrames
• tqdm: For progress tracking during benchmark execution

Quick Start

There are 3 ways to benchmark with easybench:

1. @bench decorator

   from easybench import bench
   
   # Add @bench with function parameters
   @bench(item=123, big_list=lambda: list(range(1_000_000)))
   def insert_first(item, big_list):
       big_list.insert(0, item)
   

   • When you need fresh data for each trial, use a function or lambda to generate new data on demand.
     (like lambda: list(range(1_000_000)) in the above)

2. EasyBench class

   from easybench import EasyBench, BenchConfig
   
   class BenchListOperation(EasyBench):
       # Benchmark configuration
       bench_config = BenchConfig(
           trials=10,
           memory=True,
           sort_by="avg"
       )
   
       # Setup for each trial
       def setup_trial(self):
           self.big_list = list(range(1_000_000))
   
       # Benchmark methods (must start with bench_)
       def bench_insert_first(self):
           self.big_list.insert(0, 123)
   
       # You can define multiple benchmark methods
       def bench_pop_first(self):
           self.big_list.pop(0)
   
   if __name__ == "__main__":
       # Run benchmark
       BenchListOperation().bench()
   

3. easybench command

   1. Create a benchmarks directory

   2. Put bench_*.py scripts in the directory:

      from easybench import fixture
      
      # Fixture for each trial
      @fixture(scope="trial")
      def big_list():
          return list(range(1_000_000))
      
      # Benchmark functions (must start with bench_)
      def bench_insert_first(big_list):
          big_list.insert(0, 123)
      
      # You can define multiple benchmark functions
      def bench_pop_first(big_list):
          big_list.pop(0)
      

   3. Run easybench command

      easybench --trials 10 --memory --sort-by avg
      

Example of benchmark results:

• Single benchmark

  Benchmark Results (5 trials):
  
  Function        Avg Time (s)  Min Time (s)  Max Time (s)
  --------------------------------------------------------
  insert_first        0.001568      0.001071      0.003265
  

• Multiple benchmarks

  

• Boxplot Visualization

  

• Violinplot Visualization

  

• Lineplot Visualization

  

• Histogram Visualization

  

• Barplot Visualization

  

Usage

For detailed usage instructions, please refer to the documentation.

Memory Measurement Limitations

EasyBench uses Python's built-in tracemalloc module to measure memory usage.
This has some important limitations:

• tracemalloc only tracks memory allocations made through Python's memory manager
• Memory allocated by C extensions (like NumPy, Pandas, or other native libraries) often bypasses Python's memory manager and won't be accurately measured
• The reported memory usage reflects Python objects only, not the total process memory consumption For applications heavily using C extensions, consider using external profilers like memory_profiler or system monitoring tools for more accurate measurements.

License

MIT