A simpe benchmarking tool
Project description
Simple performance measurement tool
Note: this package is still in early developpement phase, the API may change in the future. Feel free to open an issue for any comments or feedback.
Installation
neurtu requires Python 2.7 or 3.4+, it can be installed with,
pip install git+https://github.com/symerio/neurtu.git
pandas is an optional (but highly recommended) dependency.
Quickstart
Single benchmark
Measuring the run time,
>>> from neurtu import timeit, delayed >>> timeit(delayed(sum)(range(100000)), repeat=3) {'wall_time_max': 0.0018, 'wall_time_mean': 0.0017, 'wall_time_min': 0.0015, 'wall_time_std': 0.00011}
which will internally cll the timeit.Timer class. Similarly to IPython’s %timeit, the number of runs will be determined at runtime to mitigate the finite resolution of the timer (on Windows it’s 16 ms!). In addition, each evaluation will be here repeated 3 times (default) to measure statistics.
Similarity, the memory use can be measured with,
>>> from neurtu import memit, delayed >>> memit(delayed(sorted)(list(range(100000)))) {'peak_memory_max': 0.765, 'peak_memory_mean': 0.757, 'peak_memory_min': 0.753, 'peak_memory_std': 0.00552}
Generic benchmarks
Both timeit and memit are aliases for the Benchmark class which can be used for
from neurtu import Benchmark, delayed Benchmark(wall_time=True, peak_memory=True)( delayed(sorted)(range(100000)), number=3) currently supported metrics are ``wall_time``, ``peak_memory`` as well as ``cpu_time`` (Linux and Mac OS only).
Parametric benchmarks
The timeit, memit and Benchmark also accept as input sequence of delayed objects, tagged with the tag parameter,
>>> from neurtu import timeit, delayed
>>> timeit(delayed(sorted, tags={'N': N})(range(N)) for N in [1000, 10000, 100000])
whill will produce a pandas.DataFrame with the measures if pandas is installed and a list of dictionaries otherwise.
Delayed evaluation
The delayed function follows the dask.delayed API, though with a significantly lighter functionality: it models operations as a chained list of delayed objects that are not evaluated untill the compute() method is called.
>>> from neurtu import delayed
>>> x = delayed('some string').split(' ')[::-1]
>>> x
<Delayed('some string')>
->.split
->( )
->[slice(None, None, -1)]
>>> x.compute()
['string', 'some']
Attrubute access, indexing as well as function and method calls are supported. Left function composition (e.g. func(delayed(obj))) and binary operations (e.g. delayed(op) + 1) are currently not supported, neither is the composition of multiple delayed objects, use dask.delayed for it.
Scientific computing usage
A typical use case, occurs when manipulating objects with a scikit-learn API,
res = Benchmark(wall_time=True, cpu_time=True)(
delayed(NearestNeighbors, tags={'n_jobs': n_jobs})(n_jobs=n_jobs).fit(X)
for n_jobs in range(1, 10))
Motivation
The API was strongly inspired by joblib.Parallel.
The package name was taken from the Basque word neurtu meaning “to measure / evaluate”.
License
neurtu is released under the 3-clause BSD license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for neurtu-0.1a0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d60157cdbd28c8bae213215eec8264a04d13ee84de7faf36176269131792032 |
|
MD5 | 03915b78cf243f9985e897d1a235f5b1 |
|
BLAKE2b-256 | 4289ee9f1b712421b5fd43dcea3cbaba1666b73f71c57ce030f88ffef6fe822f |