im-data-manager-job-utilities 1.3.0

Job execution utilities

Informatics Matters Data Manager Job Utilities

A Python 2/3 package that simplifies the generation of events and cost lines written to a Squonk2 Job stdout stream.

The following utilities are available: -

• DmLog.emit_event()

• DmLog.emit_cost()

A number of miscellaneous utilities are also included. These have been extracted form the squonk2 virtual-screening utils.py module and moved here into utils.py.

Installation (Python)

The Job utilities are published on PyPI and can be installed from there:

pip install im-data-manager-job-utilities

Once installed you can use the available classes:

>>> from dm_job_utilities.dm_log import DmLog
>>> DmLog.emit_event('Hello World!')
2022-02-03T16:39:27+00:00 # INFO -EVENT- Hello World!
>>> from decimal import Decimal
>>> DmLog.emit_cost(Decimal('5.7'))
2022-02-03T16:40:16+00:00 # INFO -COST- 5.7 1

Costs are total by default but can be issued as an incremental value:

>>> DmLog.emit_cost(Decimal('0.5'), incremental=True)
2022-02-03T16:40:16+00:00 # INFO -COST- +0.5 2

The final value on each cost line is a unique sequence number. This value is typically an integer that increments with each line. It is required and is used by the Data Manager to avoid duplicating costs.

The Data Manager uses the python-dateutil package to parse dates and times from the generated log lines. The package is extremely tolerant of many formats but if you are in control of the string, the preferred format would be to use a classic UTC ISO string like %Y-%m-%dT%H:%M:%S%z so that the date and time is written as:

2022-03-20T10:49:41+00:00

Disabling logging

You can disable the event and cost logging by defining the following environment variables (any value will do, the presence of the variable is enough to disable the corresponding facility):

DMLOG_EVENT_DISABLE
DMLOG_COST_DISABLE

Get in touch

• Report bugs, suggest features or view the source code on GitHub.