mfethuls

From scattered instrument files to analysis-ready data. mfethuls is a Python framework that reads, cleans and organises experimental data: raw instrument exports are ingested against an experiment registry, normalised to a canonical schema and stored as traceable, queryable Parquet datasets. Use it from the app (mfethuls-app, no code needed), the command line (mfethuls), notebooks, or SQL. Experimentalists get consistent, comparable data and publication-ready figures; data scientists get harmonised data from every instrument to build features and models on.

Instrument support

Other instruments that export text or Excel files can be added from the app without code: see adding an instrument.

Each instrument's reader normalises raw exports to a canonical column schema — same column names and units regardless of instrument model.

Install

Not a Python user? Follow docs/guides/local_setup.md: one small tool (uv) installs everything, Python included, and the app then starts with a double-click.

With Python (3.11 or newer): install mfethuls in a virtual environment, so it and its dependencies stay separate from your other Python projects:

python -m venv .venv
.venv\Scripts\activate          # Windows; on macOS/Linux: source .venv/bin/activate
pip install "mfethuls[viz,notebook]"

Or with uv, which creates the environment for you: uv tool install "mfethuls[viz]" makes the mfethuls and mfethuls-app commands available everywhere.

The latest development version: pip install "mfethuls[viz,notebook] @ git+https://github.com/lucaAyt/mfethuls.git". The extras:

Getting started

No Docker, no database, no server. Start the app:

mfethuls-app

The welcome page offers three ways in:

• Get started: choose your data folder (one subfolder per instrument) and your registry (CSV or Excel, one row per experiment; the app can create an empty one or give you an example). They are saved in ~/.mfethuls/settings.json, so notebooks use the same folders; change them any time under ⚙ Settings.
• Try the example: explore example DSC, TGA, FTIR and fluorescence data, copied to ~/mfethuls-example. Your own data is not touched.
• Use it from Python: the tutorial notebook (below).

A getting-started checklist then walks through registering (Ingest → Review & register), ingesting, selecting and plotting (Colour by, Style by, axis limits, SVG/HTML export) and cleaning up (⚙ Settings → Clean up); a compact tracker stays at the top of the sidebar, next to 🏠 (back to the welcome page) and ⚙.

Non-technical users: double-click launch.bat (Windows) or run bash launch.sh; it needs only uv. See docs/guides/local_setup.md. Folders can also come from .env (copy env_example); folders saved in the app take precedence. experiments_template.csv in the repo root has an example row for every supported instrument.

Python API: a guided tutorial (a marimo notebook on its own copy of the example data) opens with mfethuls-tutorial, or Use it from Python on the app's welcome page.

import mfethuls

# Load one or more experiments
cs = mfethuls.load_experiments(["exp_001", "exp_002"])
df = cs.to_dataframe()   # tidy long-format DataFrame

# Load all experiments for a sample
cs = mfethuls.load_samples(["S001", "S002"])

Command line: the same tasks as the app, for terminals and scripts. See docs/guides/command_line.md.

mfethuls settings --data-folder D:\lab-data --registry D:\lab-data\experiments.xlsx
mfethuls validate --check-files
mfethuls ingest --all
mfethuls plot exp_001 exp_002 -o comparison.svg
mfethuls cleanup --apply

Plotting:

fig, ax = mfethuls.plot_experiments(cs)                  # publication figure (Matplotlib)
fig = mfethuls.plot_experiments(cs, backend="plotly")    # interactive, every data point (WebGL)
mfethuls.set_plot_backend("plotly")                      # make Plotly the default

Each instrument gets a suitable plot, with axis labels and units (Temperature (°C), Wavenumber (cm⁻¹)). Experiments that share an x-axis are overlaid, otherwise each gets its own panel, and time-resolved spectra series are coloured by elapsed time. Matplotlib figures use a fixed publication style and save as vector SVG/PDF. See docs/guides/plotting.md.

For teams

A shared lab server (API, background ingest, Postgres, cloud storage, OneDrive sync and a shared app) is developed separately as mfethuls-server, which builds on this package.

Data access for analysis

Ingested data is stored as one Parquet file per experiment (with a .metadata.json sidecar), in the storage folder (_storage inside the data folder by default). A DuckDB catalog (mfethuls.duckdb) lists every dataset and has one SQL view per dataset over its Parquet file. Load data through the Python API, query it with SQL (mfethuls.storage.catalog_path() gives the catalog file), or read the Parquet files directly. See docs/guides/data_analysis.md for SQL recipes and model-building patterns.

Docs

Guides

How to do things, step by step.

Reference

What each format and part is, to look things up.

License

Apache License 2.0 — see LICENSE and NOTICE. Versions released before the change were published under the MIT License.

Contributions are welcome; see CONTRIBUTING.md. First-time contributors are asked to sign the Contributor License Agreement.