Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Python module to generate and modify perf

Project Description

The Python perf module is a toolkit to write, run and analyze benchmarks.

Features

  • Simple API to run reliable benchmarks
  • Automatically calibrate a benchmark for a time budget.
  • Spawn multiple worker processes.
  • Compute the mean and standard deviation.
  • Detect if a benchmark result seems unstable.
  • JSON format to store benchmark results.
  • Support multiple units: seconds, bytes and integer.

Usage

To run a benchmark use the perf timeit command (result written into bench.json):

$ python3 -m perf timeit '[1,2]*1000' -o bench.json
.....................
Mean +- std dev: 4.22 us +- 0.08 us

Or write a benchmark script bench.py:

#!/usr/bin/env python3
import perf

runner = perf.Runner()
runner.timeit("sorted(list(range(1000)), key=lambda x: x)",
              stmt="sorted(s, key=f)",
              setup="f = lambda x: x; s = list(range(1000))")

Run the script using (result written into bench.json):

$ python3 bench.py -o bench.json

To analyze benchmark results use the perf stats command:

$ python3 -m perf stats bench.json
Total duration: 29.2 sec
Start date: 2016-10-21 03:14:19
End date: 2016-10-21 03:14:53
Raw value minimum: 177 ms
Raw value maximum: 183 ms

Number of calibration run: 1
Number of run with values: 40
Total number of run: 41

Number of warmup per run: 1
Number of value per run: 3
Loop iterations per value: 8
Total number of values: 120

Minimum:         22.1 ms
Median +- MAD:   22.5 ms +- 0.1 ms
Mean +- std dev: 22.5 ms +- 0.2 ms
Maximum:         22.9 ms

  0th percentile: 22.1 ms (-2% of the mean) -- minimum
  5th percentile: 22.3 ms (-1% of the mean)
 25th percentile: 22.4 ms (-1% of the mean) -- Q1
 50th percentile: 22.5 ms (-0% of the mean) -- median
 75th percentile: 22.7 ms (+1% of the mean) -- Q3
 95th percentile: 22.9 ms (+2% of the mean)
100th percentile: 22.9 ms (+2% of the mean) -- maximum

Number of outlier (out of 22.0 ms..23.0 ms): 0

There’s also:

  • perf compare_to command tests if a difference is significant. It supports comparison between multiple benchmark suites (made of multiple benchmarks)

    $ python3 -m perf compare_to py2.json py3.json --table
    +-----------+---------+------------------------------+
    | Benchmark | py2     | py3                          |
    +===========+=========+==============================+
    | timeit    | 4.70 us | 4.22 us: 1.11x faster (-10%) |
    +-----------+---------+------------------------------+
    
  • perf system tune command to tune your system to run stable benchmarks.

  • Automatically collect metadata on the computer and the benchmark: use the perf metadata command to display them, or the perf collect_metadata command to manually collect them.

  • --track-memory and --tracemalloc options to track the memory usage of a benchmark.

Release History

Release History

This version
History Node

1.4

History Node

1.3

History Node

1.2

History Node

1.1

History Node

1.0

History Node

0.9.6

History Node

0.9.5

History Node

0.9.4

History Node

0.9.3

History Node

0.9.2

History Node

0.9.1

History Node

0.9.0

History Node

0.8.3

History Node

0.8.2

History Node

0.8.1

History Node

0.8.0

History Node

0.7.12

History Node

0.7.11

History Node

0.7.10

History Node

0.7.9

History Node

0.7.8

History Node

0.7.7

History Node

0.7.6

History Node

0.7.5

History Node

0.7.4

History Node

0.7.3

History Node

0.7.2

History Node

0.7.1

History Node

0.7

History Node

0.6

History Node

0.5

History Node

0.4

History Node

0.3

History Node

0.2

History Node

0.1

History Node

0.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
perf-1.4-py2.py3-none-any.whl (86.6 kB) Copy SHA256 Checksum SHA256 py2.py3 Wheel Jul 6, 2017
perf-1.4.tar.gz (171.6 kB) Copy SHA256 Checksum SHA256 Source Jul 6, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting