easybench 0.1.35

A simple and easy-to-use Python benchmarking library

EasyBench

Docs

A simple and easy-to-use Python benchmarking library.

Features

• Four benchmarking styles: decorator, class-based, command-line, and Jupyter Notebook
• Measure both execution time and memory usage (see limitations)
• Rich visualizations (Boxplot, Violinplot, Lineplot, Histogram, Barplot)
• Advanced options: warmup runs, multiple loops per trial, outlier trimming
• Parametrized benchmarks for comparing function performance with different inputs
• pytest-like fixtures and lifecycle hooks (setup/teardown)
• Flexible configuration: time/memory units, progress tracking, filtering
• Multiple output formats (text tables, CSV, JSON, pandas.DataFrame)
• Extensible reporting system for custom outputs

Installation

pip install easybench

Optional Dependencies

EasyBench supports optional dependencies for additional features:

# Install with visualization support
pip install easybench[all]

The all option includes:

• matplotlib: For visualization and plotting benchmark results
• seaborn: For enhanced statistical visualizations
• pandas: For outputting benchmark results as DataFrames
• tqdm: For progress tracking during benchmark execution

Quick Start

There are 4 ways to benchmark with easybench:

1. @bench decorator

   from easybench import bench
   
   # Add @bench with function parameters
   @bench(item=123, big_list=lambda: list(range(1_000_000)))
   def insert_first(item, big_list):
       big_list.insert(0, item)
   

   • When you need fresh data for each trial, use a function or lambda to generate new data on demand.
     (like lambda: list(range(1_000_000)) in the above)

2. EasyBench class

   from easybench import EasyBench, BenchConfig
   
   class BenchListOperation(EasyBench):
       # Benchmark configuration
       bench_config = BenchConfig(
           trials=10,
           memory=True,
           sort_by="avg"
       )
   
       # Setup for each trial
       def setup_trial(self):
           self.big_list = list(range(1_000_000))
   
       # Benchmark methods (must start with bench_)
       def bench_insert_first(self):
           self.big_list.insert(0, 123)
   
       # You can define multiple benchmark methods
       def bench_pop_first(self):
           self.big_list.pop(0)
   
   if __name__ == "__main__":
       # Run benchmark
       BenchListOperation().bench()
   

3. easybench command

   1. Create a benchmarks directory

   2. Put bench_*.py scripts in the directory:

      from easybench import fixture
      
      # Fixture for each trial
      @fixture(scope="trial")
      def big_list():
          return list(range(1_000_000))
      
      # Benchmark functions (must start with bench_)
      def bench_insert_first(big_list):
          big_list.insert(0, 123)
      
      # You can define multiple benchmark functions
      def bench_pop_first(big_list):
          big_list.pop(0)
      

   3. Run easybench command

      easybench --trials 10 --memory --sort-by avg
      

4. %%easybench magic command (Jupyter Notebook)

   1. Load the extension

      %load_ext easybench
      

   2. Benchmark cell code

      %%easybench --trials=3 --memory
      result = []
      for i in range(1_000_000):
          result.append(i)
      

Example of benchmark results:

• Single benchmark

  Benchmark Results (5 trials):
  
  Function        Avg Time (s)  Min Time (s)  Max Time (s)
  --------------------------------------------------------
  insert_first        0.001568      0.001071      0.003265
  

• Multiple benchmarks

  

• Boxplot Visualization

  

• Violinplot Visualization

  

• Lineplot Visualization

  

• Histogram Visualization

  

• Barplot Visualization

  

Usage

For detailed usage instructions, please refer to the documentation.

Memory Measurement Limitations

EasyBench uses Python's built-in tracemalloc module to measure memory usage.
This has some important limitations:

• tracemalloc only tracks memory allocations made through Python's memory manager
• Memory allocated by C extensions (like NumPy, Pandas, or other native libraries) often bypasses Python's memory manager and won't be accurately measured
• The reported memory usage reflects Python objects only, not the total process memory consumption For applications heavily using C extensions, consider using external profilers like memory_profiler or system monitoring tools for more accurate measurements.

License

MIT