hairl-fer-py

Emissivity processing for the HAIRL emissometer, built on the FER data model.

The package turns raw Bruker OPUS acquisitions into spectral, total directional and total hemispherical emissivity, carrying a full uncertainty budget (GUM or Monte Carlo) through every step.

pip install hairl-fer-py

Python 3.10+. Everything is driven from one command, hairl-fer-py.

The pipeline

Processing runs as a chain of stages. Each stage reads what the previous one wrote under the experiment folder, so you can stop, inspect the intermediate FER documents, and resume.

  raw OPUS files
        │
        ▼
   ┌─────────┐
   │  init   │  lay out the experiment folder
   └─────────┘
        │  <exp>/raw/ , <exp>/configs/
        ▼
   ┌─────────┐
   │ import  │  OPUS + InfluxDB → FER          ← only stage that needs the lab PC
   └─────────┘
        │  dataset/input_quantities/
        ▼
   ┌─────────┐
   │ reduce  │  group repeats, average them
   └─────────┘
        │  dataset/processed_input_quantities/
        ▼
   ┌──────────────────┐
   │ prepare-spectral │  one config per (sample, T, angle)
   └──────────────────┘
        │  dataset/spectral_emissivity/<sample>/T_*/A_*/config.yaml
        ▼
   ┌──────────────────┐        ┌─────────┐
   │ compute-spectral │◄──────►│ inspect │  interactive tuning of one node
   └──────────────────┘        └─────────┘
        │  …/results/spectral_emissivity.json  (+ CSVs, figures)
        ▼
   ┌────────────────────┐
   │ prepare-integrated │  one config per (sample, T)
   └────────────────────┘
        │  dataset/integrated_emissivities/<sample>/T_*/config.yaml
        ▼
   ┌────────────────────┐
   │ compute-integrated │  total directional + hemispherical emissivity
   └────────────────────┘

A typical run:

hairl-fer-py init               ./my_experiment
hairl-fer-py import             ./my_experiment      # on the lab PC
hairl-fer-py reduce             ./my_experiment
hairl-fer-py prepare-spectral   ./my_experiment
hairl-fer-py compute-spectral   ./my_experiment
hairl-fer-py prepare-integrated ./my_experiment
hairl-fer-py compute-integrated ./my_experiment

Every stage except inspect and compare takes the experiment folder — the one holding init.yaml — as its only positional argument.

Commands

init <experiment>

Lays out an experiment folder from raw OPUS datasets. Reads init.yaml, copies the sample and the three blackbody reference datasets into <exp>/raw/, groups the sample files by temperature, writes one config per dataset into <exp>/configs/, and copies the FER templates in.

Sample filenames are parsed as {sample_name}_{temperature}_{angle}; a legacy layout with one numeric subdirectory per temperature is also recognised.

import <experiment>

Converts raw OPUS files into FER input quantities in dataset/input_quantities/. For each acquisition it reads the interferogram (averaging the forward and backward sweeps when both are present), then pulls the channels recorded during that acquisition window from the InfluxDB instance on the lab PC.

This is the only stage that needs the lab PC. It queries http://localhost:8086. Once dataset/input_quantities/ has been produced and exported, every later stage runs on any machine.

reduce <experiment>

Groups the input quantities by the states named in each config's processing.gather_measurements_by, then computes the mean interferogram and the mean temperatures for each group. Writes dataset/processed_input_quantities/.

prepare-spectral <experiment>

Scans the processed folders, groups them by sample, temperature and angle, and writes one config.yaml per computation node under dataset/spectral_emissivity/<sample>/T_<T>/A_<A>/.

Edit those configs to tune a node before computing it — spectral limits, uncertainty.method, the Q-factor options — or use inspect to do it interactively.

compute-spectral <experiment>

Computes spectral emissivity for every prepared node: Q factor → estimated temperatures → radiances → emissivity with its uncertainty budget. Exports CSVs and overview figures when it finishes.

A node's uncertainty.method selects gum (default, analytic sensitivity coefficients) or monte_carlo (JCGM 101 propagation of distributions).

Temperature uncertainty. The temporal component of each temperature's budget is the dispersion of the logged readings, s, not the standard error of their mean, s/√n. The temperature drifts across the acquisition window the spectrum is integrated over, so it is not a constant measurand observed n times: sampling the thermocouple more often cannot shrink that spread towards zero. The readings are serially correlated too, so n overstates the number of independent observations either way. See MeanTemperature.compute_time_standard_uncertainty.

export-spectral <experiment>

Re-exports the spectral results as per-angle CSVs, a combined CSV and an overview figure per temperature node. compute-spectral already does this; run it on its own to refresh exports without recomputing.

prepare-integrated <experiment>

Finds the nodes that have a spectral_emissivity.json, groups them by sample and temperature, and writes one config.yaml per group under dataset/integrated_emissivities/. Edit the per-angle integration limits there before computing.

compute-integrated <experiment>

Integrates the spectral emissivity over wavelength for each angle to get the total directional emissivity, then over angle to get the total hemispherical emissivity.

inspect <node>

Opens a matplotlib window for a single node folder (…/spectral_emissivity/<sample>/T_<T>/A_<A>). Adjust the spectral limits, overlay reference curves from CSV or FER JSON, re-run Monte Carlo, and write the choices back to the node's config.yaml.

compare <gum_dir> <mc_dir>

Plots two spectral_emissivity.json results against each other and reports the $E_n$ number and uncertainty ratio over a band. Pair it with compute-spectral --output-suffix.

Experiment folder layout

my_experiment/
├── init.yaml                       ← you write this (copy the packaged example)
├── configs/                        ← init writes these, you tune them
│   ├── sample_T_500.0.yaml
│   ├── high_T_bb.yaml
│   ├── low_T_bb-sample.yaml
│   └── low_T_bb-bb.yaml
├── fer_templates/                  ← copied in by init
├── raw/
│   ├── sample/T_500.0/…
│   ├── high_T_bb/…
│   ├── low_T_bb-sample/…
│   └── low_T_bb-bb/…
└── dataset/
    ├── input_quantities/           ← import
    ├── processed_input_quantities/ ← reduce
    ├── spectral_emissivity/        ← prepare-spectral, compute-spectral
    │   └── <sample>/T_*/A_*/
    │       ├── config.yaml
    │       ├── input_quantities/
    │       └── results/
    └── integrated_emissivities/    ← prepare-integrated, compute-integrated
        └── <sample>/T_*/

Re-running init is safe for configs, additive for raw data. Configs you have edited are skipped, not overwritten. The raw reference datasets are copied again into suffixed folders (high_T_bb_001, T_200.0_001, …) rather than replaced, so nothing is ever destroyed — but the copies are unused and accumulate. Delete them, or re-init into a clean folder.

Package layout

src/hairl_fer_py/
├── cli.py                      one module-level function per command
├── assets/                     32 FER templates + 7 configs, shipped in the wheel
└── core/
    ├── assets.py               asset lookup (project folder → packaged fallback)
    ├── utils.py                YAML/FER I/O helpers
    ├── opus/opus_to_fer.py     OPUS + InfluxDB → FER documents
    ├── processing/             grouping, mean interferograms, mean temperatures
    ├── emissivity/             the physics
    │   ├── q_factor.py             Q factor, magnitude and signed paths
    │   ├── radiances.py            Planck radiances and ∂L/∂T
    │   ├── estimated_temperatures.py
    │   ├── spectral_emissivity.py      GUM budget
    │   ├── spectral_emissivity_mc.py   Monte Carlo budget (JCGM 101)
    │   ├── total_directional_emissivity.py
    │   ├── total_hemispherical_emissivity.py
    │   └── pipeline_executor.py    runs one node end to end
    ├── pipeline/               one module per CLI command
    └── tools/                  inspector, GUM-vs-MC comparison

Each pipeline module is named after the command that calls it — compute-spectral lives in core/pipeline/compute_spectral.py.

Dependencies

Two of these are maintained alongside this package:

The floors are ferpy>=0.2.0 and ft4ftirs>=1.1.0, and neither is negotiable. ferpy 0.2.0 fixes FER document round-tripping, which this package does at every stage; ft4ftirs 1.1.0 fixes zero filling, interferogram x-spacing and forward/backward handling, all on paths import and compute-spectral use. Earlier versions produce different numbers.

ft4ftirs replaces the earlier ftirpy; there is no compatibility shim.

Development

pip install -e ".[dev]"
pytest

The version comes from the git tag via setuptools-scm — there is no version string in the source tree. A build from an untagged or dirty checkout is labelled X.Y.Z.devN+g<sha>, so it is always distinguishable from a release. Building from an export with no .git fails by design; set SETUPTOOLS_SCM_PRETEND_VERSION if you need to.

The integration marker covers the Monte Carlo worker loop, which is slower:

pytest -m integration

License

Copyright (C) 2026 Jon Gabirondo-López

hairl-fer-py is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

It is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.