uhammer 0.4.0

Convenience layer for emcee sampler

uhammer offers a convenience layer for emcee.

Features: uhammer

• offers a simplified API.

• requires no code changes between running on multiple cores or with MPI.

• fixes some issues with the MPI Pool from emcee / schwimmbad.

• prints diagnostic messages when allocated nodes / cores do not fit well to specified number of walkers or other parallelization related settings.

• can capture worker specific output to separate files.

• implements persisting of sampler state and supports continuation of sampling at a later time.

• can show an animated progress bar.

Example usage

To use uhammer you need:

• an instance of Parameters for declaring the parameters you want to sample from.

• a function, e.g. named lnprob, which takes a parameters object and possible extra arguments. This function returns the logarithic value of the computed posterior probability.

• finally you call sample for running the sampler.

import time

import numpy as np

from uhammer import Parameters, sample

sigma = .5


def gen_data():
   a0 = .5
   b0 = .5
   c0 = 1

   x = np.linspace(-2, 2, 100)
   y_measured = a0 + b0 * x + c0 * x ** 2 + sigma * np.random.randn(*x.shape)
   return x, y_measured


p = Parameters()
p.add("a", (0, 1))
p.add("b", (0, 1))
p.add("c", (0, 2))


def lnprob(p, x, y_measured):
   time.sleep(.0002)
   y = p.a + p.b * x + p.c * x ** 2
   diff = (y - y_measured) / sigma
   return -np.dot(diff.T, diff) / 2


n_samples = 15000
n_walkers_per_param = 200


samples, lnprobs = sample(
   lnprob,
   p,
   args=gen_data(),
   n_walkers_per_param=n_walkers_per_param,
   n_samples=n_samples,
   show_progress=True,
   show_output=False,
)

print()
print(samples[5000:].mean(axis=0))

$ python examples/sample_line_fit.py
uhammer: perform 25 steps of emcee sampler
✗ passed: 00:00:11.2 left: 00:00:00.0 - [∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣]

[0.52389808 0.53415134 1.01585175]

Credits

This package was created with Cookiecutter and the uweschmitt/cookiecutter-pypackage project template.

History

0.4.0 (2019-07-03)

• Ignore start values for parameters. Did not play well with the emcee algorithm.

0.3.2 (2019-05-24)

• Less chatty and simpler progress report on euler. The existing progressbar cluttered the lsf output files. Now we report in min time intervals of at least 30 seconds.

0.3.1 (2019-05-22)

• removed mpipool from installation dependencies, else installation always needs a working MPI setup. Still mpirpool is needed when run in parallel MPI mode.

0.3.0 (2019-05-22)

• sample functions now return two arrays: the samples and the related log probabilities

• Parameters.add now has an optional argument for providing a starting value.

• load_samples function allows extracting samples from a persisted sampler file.

• Replaced capture_output argument of sample and continue_sampling by two arguments show_output and output_prefix.

• Fixed bug in output recording in continue_sampling function.

• Use mpipool librariy now + added test for mpi based parallel sampling.

0.2.13 (2019-05-15)

• fix persisting pickler error in parallel mode

0.2.12 (2019-05-02)

• minor tweaks for mpi

• fix for emcee 3 dev version

0.2.11 (2019-04-30)

• shutdown mpi pool in case a worker throws an exception, before this fix mpirun would hang.

• fix handling of capture output file names when running with mpi

0.2.10 (2019-04-19)

• fix race condition when removing marker file

• deactivate unnecessary warnings about missing mpi4py

0.2.9 (2019-04-16)

• Don’t show progressbar when run on euler node

• Works with Python 3.7

0.2.8 (2019-04-12)

• Fix to work with dev version of emcee 3

0.2.7 (2019-03-20)

• Fix issue with capuring output in parallel mode

• Stop progress bar in case of unhandled exception

0.2.6 (2019-03-20)

• Fix dependencies for Python 3.7 in setup.py

0.2.5 (2019-03-20)

• Fix package lookup in requirements_dev.txt

• Fix error in error handling when output redirection fails.

0.2.4 (2018-10-29)

• fix ordering of sampler output rows.

0.2.3 (2018-10-24)

• check OMP_NUM_THREADS to warn of possible over subscription.

• better pickling support for posterior function.

0.2.2 (2018-10-11)

• Fix detection if uhammer runs on full node on euler.

• Dont show statusbar if direct write to fid 1 is not possible.

• Supress some unappropriate error messages from MPI, even if uhammer is not run using mpirun.

• Fix ip check to detect if uhammer runs on euler.

0.2.1 (2018-09-28)

• Fix of regression due to implementation of Python 2 support.

0.2.0 (2018-09-28)

• Introduced Python 2 support.

0.1.0 (2018-08-23)

• Initial version.