benchit 0.0.4

Benchmarking tools for Python

Tools to benchmark Python solutions on runtime performance and visualize. Based on timeit, it primarily aims to functionally simulate the timeit behaviour and hence the name! This facilitates benchmarking on multiple datasets and solutions.

Documentation

Installation

Latest PyPI stable release :

pip install benchit

Pull latest development release on GitHub and install in the current directory :

pip install -e git+https://github.com/droyed/benchit.git@master#egg=benchit

Getting started

Consider a setup to compare NumPy ufuncs - sum, prod, max on arrays varying in their sizes. To keep it simple, let’s consider 1D arrays. Thus, we would have :

>>> import numpy as np
>>> funcs = [np.sum,np.prod,np.max]
>>> inputs = [np.random.rand(i) for i in 10**np.arange(5)]

>>> import benchit
>>> t = benchit.timings(funcs, inputs)

It’s a dataframe-like object and as such we can plot it. It automatically adds in specs into the title area to convey all of available benchmarking info :

>>> t.plot(logy=True, logx=True)

More realistic example

Let’s consider a setup where functions accept more than one argument. Let’s take the case of computing euclidean distances between two 2D arrays. We will feed in arrays with varying number of rows and 3 columns to represent data in 3D Cartesian coordinate system and benchmark two commonly used functions in Python.

>>> from sklearn.metrics.pairwise import pairwise_distances
>>> from scipy.spatial.distance import cdist
>>> fns = [cdist, pairwise_distances]

>>> import numpy as np
>>> in_ = {n:[np.random.rand(n,3), np.random.rand(n,3)] for n in [10,100,500,1000,4000]}
>>> t = benchit.timings(fns, in_, multivar=True, input_name='Array-length')
>>> t.plot(logx=True)

Plotting on notebooks?

Simply use benchit.set_environ(‘notebook’) before plotting.

Check out sample notebook run.