Skip to main content

Utility package to run CHM jobs on clusters

Project description

chmutil

https://travis-ci.org/CRBS/chmutil.svg?branch=master https://coveralls.io/repos/github/CRBS/chmutil/badge.svg?branch=master

Utility package to run Cascaded Hierarchical Model (CHM) jobs on clusters.

For more information visit our wiki page

Tools

  • createchmjob.py – Creates a set of CHM jobs to process a set of images on various compute clusters (Gordon, Comet, & Rocce)
  • checkchmjob.py – Checks status of jobs generated by createchmjob.py script
  • createchmtrainjob.py – Creates a CHM train job, runnable on various compute clusters (Comet & Rocce), which teaches CHM how to segment a desired feature. The output from this job is a trained model that is needed as an input for createchmjob.py
  • createchmimage.py – Simple image conversion utility
  • createprobmapoverlay.py – Overlays semi-transparent colored probability maps over base image
  • mergetiles.py – Merges a set of image tiles in a directory into single image
  • chmrunner.py – Internal tool that runs a CHM task created by createchmjob.py
  • mergetilerunner.py – Internal tool that runs Merge task created by createchmjob.py

Dependencies

Compatibility

  • Should work on Python 2.7 & 3+ on Linux distributions

Installation

pip install chmutil

Usage

# creates a job assuming images are in ./images and ./trainedmodel has
# chm model
createchmjob.py ./images ./trainedmodel myrun --cluster rocce

License

See LICENSE.txt

Bugs

Please report them here

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.8.4 (2018-03-20)

  • Try number two to get comet configuration to load singularity 2.3.2. See issue 76 below.

0.8.3 (2018-03-16)

  • Added –dontdeletescratch option to createtrainingmrcstack.py to skip deletion of scratchdir. Issue #78
  • Updated comet configuration to load singularity 2.3.2. Issue #76

0.8.2 (2017-07-27)

  • Fixed typo in history notes which caused pypi not to render project description properly

0.8.1 (2017-07-27)

  • Fixed bug in createtrainingmrcstack.py which caused program to fail due to incorrect call to get_image_path_list(). Issue #48
  • Increased required memory for merge tasks on Rocce cluster from 10 to 20 gigabytes of ram and added virtual_free constraint to handle gigapixel images. Issue #49

0.8.0 (2017-06-23)

  • Added –gentifs to createchmjob.py to generate tif probability map images Issue #43
  • Fixed bug in checkchmjob.py –detailed where ram value was incorrect Issue #44
  • Added information about input images to checkchmjob.py –detailed Issue #28

0.7.0 (2017-06-09)

  • Added –gentiles flag to createchmimage.py to generate tiles from image that can be viewed in probability map viewer. Issue #42
  • Added –rawthreshold flag to createprobmapoverlay.py which lets caller specify exact pixel intensity value instead of percentage as offered with –threshpc. Issue #41
  • Added virtual_free to requirements for SGE jobs submitted by createchmtrainjob.py to rocce cluster. Issue #40

0.6.0 (2017-03-03)

  • Added –detailed option to checkchmjob.py. Issue #6
  • Added –addprobmap to createprobmapoverlay.py to enable additional probability maps to be overlayed. Issue #25
  • Added createchmtrainjob.py script that creates a CHM train job. Issue #19
  • Fixed incorrect script names in readme.txt generated by createchmjob.py. Issue #20

0.5.1 (2017-01-12)

  • Fixed account not being put into runjobs.gordon and runmerge.gordon. Issue #11
  • Fixed errors in checking job status examples in readme.txt created by createchmjob.py. Issue #12

0.5.0 (2017-01-11)

  • First release on PyPI.

Project details


Release history Release notifications

This version
History Node

0.8.4

History Node

0.8.2

History Node

0.8.0

History Node

0.7.0

History Node

0.6.0

History Node

0.5.1

History Node

0.5.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
chmutil-0.8.4-py2.py3-none-any.whl (83.5 kB) Copy SHA256 hash SHA256 Wheel py2.py3 Mar 21, 2018
chmutil-0.8.4.tar.gz (80.5 kB) Copy SHA256 hash SHA256 Source None Mar 21, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page