IMCtermite

IMCtermite provides access to proprietary IMC measurement file formats introduced and developed by imc Test & Measurement GmbH. The library currently supports both the legacy IMC2 Data Format and the newer IMC3 Data Format, which are commonly encountered with the file extensions .raw and .dat. These formats are employed i.a. by the measurement hardware imc CRONOSflex to dump and store data and the software packages imc Studio & imc FAMOS for measurement data control and analysis. Thanks to the integrated Python module, the extracted measurement data can be stored in any open-source file format accessible by Python like i.a. csv, json or parquet.

On the Record Evolution Platform, the library can be used both as a command line tool for interactive usage and as a Python module to integrate IMC measurement files into any ETL workflow.

Supported formats

• IMC2 marker-based files. Reference: docs/imc2-format.md
• IMC3 files, including bundled and single-channel variants. Reference: docs/imc3-format.md

Overview

• Supported formats
• Format reference
• Build and Installation
• Usage and Examples
• Testing
• Releases
• References

Format reference

IMCtermite auto-detects the file format while loading a file. The same CLI and Python interfaces are used for both supported formats; no explicit format flag is required.

Detailed format notes live in dedicated pages:

• IMC2 format reference
• IMC3 format reference

Installation

The IMCtermite library may be employed both as a CLI tool and a python module.

CLI tool

To build the CLI tool locally, use the default target make resulting in the binary imctermite. To ensure system-wide availability, the installation of the tool (in the default location /usr/local/bin) is done via

make install

which may require root permissions.

Python

To integrate the library into a customized ETL toolchain, several python targets are available. For a local in-place build that enables you to run the examples, use:

make python-build

Installation with pip

This fork is distributed as imctermite3 on the Python Package Index. Once published, install it with

python3 -m pip install imctermite3

The Python import remains from imctermite import ImcTermite. The distribution provides binary wheels for Windows and Linux with Python 3.10-3.13 and requires NumPy, which pip installs automatically.

Do not install imctermite and imctermite3 in the same environment: both provide the imctermite import package. To replace the original distribution:

python3 -m pip uninstall imctermite
python3 -m pip install imctermite3

However, if your platform/architecture is not supported you can still compile the source distribution yourself, which requires python3_setuptools, numpy, and an up-to-date compiler supporting C++11 standard (e.g. gcc version >= 10.2.0).

Usage

CLI

The usage of the imctermite binary looks like this:

imctermite <imc-file> [options]

You have to provide a single supported IMC file and any option to specify what to do with the data. All available options can be listed with imctermite --help:

Options:

 -c, --listchannels      list channels
 -b, --listblocks        list IMC key-blocks
 -d, --output            output directory to print channels
 -s, --delimiter         csv delimiter/separator char for output
 -h, --help              show this help message
 -v, --version           display version

For instance, to show a list of all channels included in sample-data.raw, you do imctermite sample-data.raw --listchannels. No output files are written by default. Output files are written only when an existing (!) directory is provided as argument to the --output option. By default, every output file is written using a , delimiter. You may provide any custom separator with the option --delimiter. For example, in order to use |, the binary is called with options imctermite sample-data.raw -b -c -s '|'.

Python

Given the IMCtermite module is available, we can import it and declare an instance of it by passing an IMC file to the constructor:

from imctermite import ImcTermite

imcraw = ImcTermite("sample/sampleA.raw")

An example of how to create an instance and obtain the list of channels is:

from imctermite import ImcTermite

# declare and initialize instance of "imctermite" by passing an IMC file
try :
    imcraw = ImcTermite("samples/sampleA.raw")
except RuntimeError as e :
    print("failed to load/parse IMC file: " + str(e))

# obtain list of channels as list of dictionaries (without data)
channels = imcraw.get_channels(False)
print(channels)

Typed channel metadata

For integrations that need a stable metadata contract without loading sample data, use the typed API:

from imctermite import ImcTermite

imcraw = ImcTermite("samples/sampleA.raw")
for channel in imcraw.get_channels_metadata():
    print(channel.name, channel.y_unit, channel.sample_count)

ChannelMetadata.name is the canonical channel name and falls back to the IMC component-group name when the source channel name is empty. The literal source value remains available as ChannelMetadata.source_name. The existing get_channels() dictionary schema is unchanged.

File, group, standalone text-object, and typed channel-property metadata are available through separate typed getters:

metadata = imcraw.get_file_metadata()
groups = imcraw.get_groups_metadata()
text_objects = imcraw.get_text_objects_metadata()
channel = imcraw.get_channels_metadata()[0]

print(metadata.producer, metadata.comment)
print(channel.properties)

TSA custom-decoder support

For native custom decoders, lib/imc_raw.hpp provides copied TSA container bytes and logical timestamped-record access. This supports application-specific frame decoding while IMCtermite remains responsible only for container parsing, metadata, and sample access; it does not decode proprietary protocol descriptions or write IMC files.

A more complete example, including the methods for obtaining the channels, i.a. their data and/or directly printing them to files, can be found in the python/examples folder.

Chunked NumPy export (fast path)

For large files, you can iterate over channel data in chunks as NumPy arrays. This avoids creating large Python lists and allows for streaming processing (e.g. writing to Parquet). See python/examples/usage_numpy_chunks.py for a complete example.

Testing

Prepare the local test environment with make prepare-test, then run end-to-end tests with make test.

The Python environment is expected to be prepared outside Make and already contain the required build and test dependencies.

To emit the Python wrapper coverage report through the same target, ensure pytest-cov is installed and use make test COVERAGE=1.

See tests/README.md for details.

References

IMC

• https://www.imc-tm.de/produkte/messtechnik-software/imc-famos/import-export
• https://www.imc-tm.de/produkte/messtechnik-hardware/imc-cronosflex/ueberblick/
• https://www.imc-tm.de/download-center/produkt-downloads/imc-famos/handbuecher
• https://www.imc-tm.de/fileadmin/Public/Downloads/Manuals/imc_FAMOS/imcGemeinsameKomponenten.pdf
• https://github.com/Apollo3zehn/ImcFamosFile
• https://apollo3zehn.github.io/ImcFamosFile/api/ImcFamosFile.FamosFileKeyType.html

Cython

• https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html

PyPI

• https://pypi.org/help/#apitoken
• https://sgoel.dev/posts/uploading-binary-wheels-to-pypi-from-github-actions/
• https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsrun
• https://github.com/pypa/cibuildwheel/blob/main/examples/github-deploy.yml
• https://cibuildwheel.readthedocs.io/en/stable/deliver-to-pypi/
• https://github.com/actions/download-artifact#download-all-artifacts
• https://github.com/actions/download-artifact?tab=readme-ov-file#download-multiple-filtered-artifacts-to-the-same-directory

iconv

• https://www.gnu.org/software/libiconv/
• https://vcpkg.io/en/packages.html
• https://vcpkg.io/en/getting-started