Skip to main content

A utility to monitor the jobs ressources in a HPC environment, espacially OAR

Project description

Colmet - Collecting metrics about jobs running in a distributed environnement


Colmet is a monitoring tool to collect metrics about jobs running in a
distributed environnement, especially for gathering metrics on clusters and
grids. It provides currently several backends :
- taskstats: fetch task metrics from the linux kernel
- stdout: display the metrics on the terminal
- zeromq: transport the metrics across the network
- hdf5: store the metrics on the filesystem


For detailed instructions on how to install Colmet on your plateform, please
refer to the INSTALL document in the same directory as this document. Please
carefully read the REQUIREMENTS section of the INSTALL instructions.


for the nodes :

sudo colmet-node -vvv -zeromq-uri tcp://

for the collector :

colmet-collector -vvv --zeromq-bind-uri tcp:// --hdf5-filepath /data/colmet.hdf5 --hdf5-complevel 9

You will see the number of counters retrieved in the debug log.

For more information, please refer to the help of theses scripts (--help)


This product is distributed under the GNU General Public License Version2.
Please read through the file LICENSE for more information about our license.


version 0.3.0:

- Divided colmet package into three parts
- colmet-node : Retrieve data from taskstats and procstats and send to
collectors with ZeroMQ
- colmet-collector : A collector that stores data received by ZeroMQ in a
hdf5 file
- colmet-common : Common colmet part.
- Added some parameters of ZeroMQ backend to prevent a memory overflow
- Simplified the command line interface
- Dropped rrd backend because it is not yet working
- Added ``--buffer-size`` option for collector to define the maximum number of
counters that colmet should queue in memory before pushing it to output
- Handled SIGTERM and SIGINT to terminate colmet properly

version 0.2.0:

- Added options to enable hdf5 compression
- Support for multiple job by cgroup path scanning
- Used Inotify events for job list update
- Don't filter packets if no job_id range was specified, especially with zeromq
- Waited the cgroup_path folder creation before scanning the list of jobs
- Added procstat for node monitoring through fictive job with 0 as identifier
- Used absolute time take measure and not delay between measure, to avoid the
drift of measure time
- Added workaround when a newly cgroup is created without process in it
(monitoring is suspended upto one process is launched)

version 0.0.1:

- Conception

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
colmet-0.3.0.tar.gz (35.4 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page