Skip to main content

No project description provided

Project description

Build status. Documentation status. PyPI package version.

Scientific software development often relies on stochasticity, e.g. for Monte Carlo integration or simulating the Ising model. Testing non-deterministic code is difficult. This package offers a bootstrap test to validate stochastic algorithms, including multiple hypothesis correction for vector statistics. It can be installed by running pip install pytest-bootstrap.


Suppose we want to implement the expected value of log-normal distribution with location parameter \mu and scale parameter \sigma.

>>> import numpy as np
>>> def lognormal_expectation(mu, sigma):
...   return np.exp(mu + sigma ** 2 / 2)
>>> def lognormal_expectation_wrong(mu, sigma):
...   return np.exp(mu + sigma ** 2)

We can validate our implementation by simulating from a lognormal distribution and comparing with the bootstrapped mean.

>>> from pytest_bootstrap import bootstrap_test
>>> mu = -1
>>> sigma = 1
>>> reference = lognormal_expectation(mu, sigma)
>>> x = np.exp(np.random.normal(mu, sigma, 1000))
>>> result = bootstrap_test(x, np.mean, reference)

This returns a summary of the test, such as the bootstrapped statistics.

>>> result.keys()
dict_keys(['alpha', 'alpha_corrected', 'reference', 'lower', 'upper',
           'z_score', 'median', 'iqr', 'tol', 'statistics'])

Comparing with our incorrect implementation reveals the bug.

>>> reference_wrong = lognormal_expectation_wrong(mu, sigma)
>>> result = bootstrap_test(x, np.mean, reference_wrong)
Traceback (most recent call last):
pytest_bootstrap.BootstrapTestError: the reference value 1.0 lies outside
  the 1 - (alpha = 0.01) interval ...

Visualising the bootstrapped distribution using pytest_bootstrap.result_hist can help identify discrepancies between the bootstrapped statistics and the theoretical reference value. Note that you need to install matplotlib separately or install pytest-bootstrap using pip install pytest-bootstrap[plot].

.. plot:: examples/
:caption: Histogram of bootstrapped means reveals the erroneous implementation of the log-normal mean.

A comprehensive set of examples can be found in the tests.


.. automodule:: pytest_bootstrap

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest-bootstrap-0.2.3.tar.gz (5.3 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page