Skip to main content

Convenience layer for emcee sampler

Project description

uhammer offers a convenience layer for emcee.

Features: uhammer

  • offers a simplified API.

  • requires no code changes between running on multiple cores or with MPI.

  • fixes some issues with the MPI Pool from emcee / schwimmbad.

  • prints diagnostic messages when allocated nodes / cores do not fit well to specified number of walkers or other parallelization related settings.

  • can capture worker specific output to separate files.

  • implements persisting of sampler state and supports continuation of sampling at a later time.

  • can show an animated progress bar.

Example usage

To use uhammer you need:

  • an instance of Parameters for declaring the parameters you want to sample from.

  • a function, e.g. named lnprob, which takes a parameters object and possible extra arguments. This function returns the logarithic value of the computed posterior probability.

  • finally you call sample for running the sampler.

import time

import numpy as np

from uhammer import Parameters, sample

sigma = .5


def gen_data():
   a0 = .5
   b0 = .5
   c0 = 1

   x = np.linspace(-2, 2, 100)
   y_measured = a0 + b0 * x + c0 * x ** 2 + sigma * np.random.randn(*x.shape)
   return x, y_measured


p = Parameters()
p.add("a", (0, 1))
p.add("b", (0, 1))
p.add("c", (0, 2))


def lnprob(p, x, y_measured):
   time.sleep(.0002)
   y = p.a + p.b * x + p.c * x ** 2
   diff = (y - y_measured) / sigma
   return -np.dot(diff.T, diff) / 2


n_samples = 15000
n_walkers_per_param = 200


samples, lnprobs = sample(
   lnprob,
   p,
   args=gen_data(),
   n_walkers_per_param=n_walkers_per_param,
   n_samples=n_samples,
   show_progress=True,
   show_output=False,
)

print()
print(samples[5000:].mean(axis=0))
$ python examples/sample_line_fit.py
uhammer: perform 25 steps of emcee sampler
✗ passed: 00:00:11.2 left: 00:00:00.0 - [∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣]

[0.52389808 0.53415134 1.01585175]

Credits

This package was created with Cookiecutter and the uweschmitt/cookiecutter-pypackage project template.

History

0.3.2 (2019-05-24)

  • Less chatty and simpler progress report on euler. The existing progressbar cluttered the lsf output files. Now we report in min time intervals of at least 30 seconds.

0.3.1 (2019-05-22)

  • removed mpipool from installation dependencies, else installation always needs a working MPI setup. Still mpirpool is needed when run in parallel MPI mode.

0.3.0 (2019-05-22)

  • sample functions now return two arrays: the samples and the related log probabilities

  • Parameters.add now has an optional argument for providing a starting value.

  • load_samples function allows extracting samples from a persisted sampler file.

  • Replaced capture_output argument of sample and continue_sampling by two arguments show_output and output_prefix.

  • Fixed bug in output recording in continue_sampling function.

  • Use mpipool librariy now + added test for mpi based parallel sampling.

0.2.13 (2019-05-15)

  • fix persisting pickler error in parallel mode

0.2.12 (2019-05-02)

  • minor tweaks for mpi

  • fix for emcee 3 dev version

0.2.11 (2019-04-30)

  • shutdown mpi pool in case a worker throws an exception, before this fix mpirun would hang.

  • fix handling of capture output file names when running with mpi

0.2.10 (2019-04-19)

  • fix race condition when removing marker file

  • deactivate unnecessary warnings about missing mpi4py

0.2.9 (2019-04-16)

  • Don’t show progressbar when run on euler node

  • Works with Python 3.7

0.2.8 (2019-04-12)

  • Fix to work with dev version of emcee 3

0.2.7 (2019-03-20)

  • Fix issue with capuring output in parallel mode

  • Stop progress bar in case of unhandled exception

0.2.6 (2019-03-20)

  • Fix dependencies for Python 3.7 in setup.py

0.2.5 (2019-03-20)

  • Fix package lookup in requirements_dev.txt

  • Fix error in error handling when output redirection fails.

0.2.4 (2018-10-29)

  • fix ordering of sampler output rows.

0.2.3 (2018-10-24)

  • check OMP_NUM_THREADS to warn of possible over subscription.

  • better pickling support for posterior function.

0.2.2 (2018-10-11)

  • Fix detection if uhammer runs on full node on euler.

  • Dont show statusbar if direct write to fid 1 is not possible.

  • Supress some unappropriate error messages from MPI, even if uhammer is not run using mpirun.

  • Fix ip check to detect if uhammer runs on euler.

0.2.1 (2018-09-28)

  • Fix of regression due to implementation of Python 2 support.

0.2.0 (2018-09-28)

  • Introduced Python 2 support.

0.1.0 (2018-08-23)

  • Initial version.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

uhammer-0.3.4.tar.gz (36.6 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page