nmrmetaproc

NMR Metabolomics Spectral Processor

nmrmetaproc converts raw Bruker NMR FID files into clean, analysis-ready spectral matrices (CSV format) suitable for PCA, PLS-DA, pathway analysis, and other downstream metabolomics workflows. It implements a rigorous, reproducible processing pipeline with automatic phase correction, chemical-shift referencing, robust baseline correction, spectral alignment, and Probabilistic Quotient Normalization (PQN).

Authors: Folorunsho Bright Omage, Toyin Bright Omage, Ljubica Tasic
ORCID: 0000-0002-9750-5034
Email: omagefolorunsho@gmail.com
Provenance: Zenodo DOI, CITATION.cff, AUTHORS.md, NOTICE, PROVENANCE.md

Maturity, and what the classifier does and does not claim. The package ships the PyPI classifier Development Status :: 5 - Production/Stable. That is a statement about the packaging and the stability of the API and the release process, over one scope: raw Bruker 1D metabolomics preprocessing and matrix export. It is not a claim that any result is validated for production use. The public benchmarks say so in their own record — every dataset in four of the five datasets in benchmarks/public_benchmark_manifest.json carry production_readiness: false inside their current_scientific_validation block, meaning the benchmark establishes agreement on that dataset and nothing wider. The fifth, the nmrglue fixture, carries no such key. Neither statement contradicts the other; they answer different questions, and both are deliberate. nmrmetaproc is not a clinical diagnostic, not a metabolite identification tool, and not a universal biological QC classifier.

Features

• Reads raw Bruker FID files (fid/*.fid + acqus/acqu) directly, no conversion needed
• Automatically discovers valid Bruker spectra under the opened directory (that folder is the study) and splits mixed data into scientifically comparable batches by pulse program
• Applies pulse-program-specific and sample-context-aware processing presets for common Bruker 1D 1H experiments (noesygppr1d including noesy1d/noesyd aliases, cpmgpr1d, zg30, ledbpgppr2s1d)
• Full processing pipeline in correct order:
  1. Exponential apodization (line broadening)
  2. Zero-filling
  3. Fast Fourier Transform
  4. Phase correction with one first-order phase per acquisition set-up (an analyst value, the spectrometer's stored phase, or pairs of isolated lines such as formate and glucose H1) and a zero-order phase per spectrum from its probe lines; each spectrum records where both phases came from and the residual phase of its lines. phase_method="acme" keeps per-spectrum entropy phasing (ACME)
  5. Context-aware chemical-shift referencing to TSP/DSS, TMS, residual CHCl3, or lipid methylene as appropriate; reference_peak="glucose" anchors serum or plasma without a usable 0 ppm standard on the alpha-glucose anomeric H1 doublet at 5.23 ppm (TSP binds plasma proteins, so its apparent shift and intensity are unreliable in whole serum)
  6. Asymmetric least-squares (ALS) baseline correction
  7. Negative-value handling with per-sample logging
  8. Solvent/context region exclusion, for example aqueous water or residual CHCl3
  9. Optional spectral alignment (one whole-spectrum correlation shift against the pointwise cohort median, or reference-peak alignment)
  10. Configurable region exclusion
  11. Uniform binning
  12. PQN normalization (default), total area, or none. Pipeline-level TSP normalization is rejected until validated pre-exclusion reference quantification is available.
• Per-sample quality control: SNR, reference-peak linewidth, solvent-region score, clean pass/warn/fail separation, and optional adaptive second-pass audit
• Clean CSV outputs ready for MetaboAnalyst, R, MATLAB, machine learning, and chemometrics
• Works on Windows, macOS, and Linux

Download

Pre-built desktop applications for Windows, macOS, and Linux are published on the Releases page. Research-beta releases are marked clearly there.

These bundles contain Python, all dependencies, and the GUI app. You do not need to install Python separately to use them.

If you prefer Python integration (CLI + scriptable API), install from PyPI instead -- see Installation below.

First-launch notes

The desktop bundles are unsigned and not notarized -- we do not yet hold an Apple Developer ID or a Windows EV code-signing certificate, so the operating system shows a warning the first time you open the app. They are built from the release tag with bundled dependencies. Before installing, verify the download against the SHA256SUMS.txt file attached to that release.

macOS (Gatekeeper) -- if you see "Apple could not verify nmrmetaproc is free of malware" with no Open button, do one of the following after dragging the .app to /Applications:

• Terminal one-liner (fastest):

  xattr -dr com.apple.quarantine /Applications/nmrmetaproc.app
  

  Then launch normally.

• GUI path: click Done on the dialog (do not click Move to Bin), then open System Settings -> Privacy & Security, scroll to the Security section, click Open Anyway next to the "nmrmetaproc was blocked" line, and re-launch the app.

Windows (SmartScreen) -- on first launch you may see "Windows protected your PC". Click More info, then Run anyway.

Linux (AppImage permissions) -- the file needs the executable bit set:

chmod +x nmrmetaproc-gui-linux-x86_64.AppImage

If your distro complains about FUSE, install libfuse2 (Ubuntu: sudo apt install libfuse2).

Installation

pip install nmrmetaproc

Or from source:

git clone https://github.com/omagebright/nmrmetaproc.git
cd nmrmetaproc
pip install -e .

Dependencies: nmrglue, numpy, scipy, pandas, tqdm

Supported Python versions are 3.9 through 3.13. Python 3.14 is excluded because the verified scientific dependency stack does not yet produce reliable numeric results there.

Try It Now - Public Benchmark Demo

Try nmrmetaproc instantly with public, attributable NMR data. The interactive Google Colab notebook installs nmrmetaproc==1.0.37 from PyPI (the same version as the CLI and desktop apps; the 1.0.37 wheel must be on PyPI before the notebook runs). The repository is private, so the badge opens a public gist copy that anonymous users can load. It does not fetch a private GitHub zip. pip install nmrmetaproc==1.0.37 is the only install step.

Demo features:

• Download public Bruker HumanSerum FIDs from PepsNMRData
• Check the published FinalSpectra_HS matrix shape
• Run NMRMetaProc and write an auditable benchmark JSON
• Show QC diagnostics without inventing source pass/fail labels
• Zero installation required - runs in your browser

Launch Demo

Command-Line Usage

Full Processing Pipeline

Point nmrmetaproc at the study directory. By default it discovers all valid spectra, treats that folder as one cohort, separates incompatible pulse programs into independent batches, applies the registered pulse-program presets, and writes one analysis-ready matrix per batch.

nmrmetaproc process /path/to/bruker/data --output ./results

To run the transparent adaptive second pass from the CLI, add:

nmrmetaproc process /path/to/bruker/data --output ./results --adaptive-second-pass

The first-pass QC remains available as qc_report_initial.csv; the selected second-pass candidate and before/after QC metrics are recorded in adaptive_qc_report.csv. A corrected pass requires an improved rerun QC with a measurable reference peak; candidates that lose reference evidence are rejected.

Batch output layout:

results/
|-- batch_manifest.csv
|-- public_serum__zg30/
|   |-- spectral_matrix.csv
|   |-- qc_report.csv
|   |-- qc_report_initial.csv              # when adaptive second pass is enabled
|   |-- adaptive_qc_report.csv             # when adaptive second pass is enabled
|   |-- acquisition_parameters.csv
|   `-- processing.log
`-- public_urine__noesygppr1d/
    |-- spectral_matrix.csv
    |-- qc_report.csv
    |-- qc_report_initial.csv              # when adaptive second pass is enabled
    |-- adaptive_qc_report.csv             # when adaptive second pass is enabled
    |-- acquisition_parameters.csv
    `-- processing.log

Use --single-matrix only when the input directory is already one homogeneous cohort and pulse program:

nmrmetaproc process /path/to/one_batch --output ./results --single-matrix

QC Scan Only

nmrmetaproc qc /path/to/data --output ./qc_results

Inspect Available Samples

nmrmetaproc info /path/to/data

Python API

from nmrmetaproc import NMRProcessor

processor = NMRProcessor(
    lb=0.3,
    bin_width=0.01,
    normalization="pqn",
    ppm_range=(0.5, 9.5),
    snr_threshold=10.0,
    linewidth_threshold=2.5,
    align="correlation",  # optional; robust cohort-median target
)

results = processor.process("/path/to/bruker/data")

print(results.spectral_matrix)   # rows=samples, columns=ppm bins
print(results.qc_report)         # SNR, linewidth, pass/fail per sample

results.save("./output")

Output Files

Data Format

Each sample must be in its own directory containing:

• fid or *.fid - binary FID data (interleaved real/imaginary int32)
• acqus or acqu - acquisition parameter file

data_root/
|-- sample_001/
|   |-- fid
|   `-- acqus
`-- sample_002/
    |-- fid
    `-- acqus

Nested layouts are also supported and discovered automatically.

Alignment contract

align="correlation" applies one integer, whole-spectrum shift with edge padding. The pipeline targets the pointwise cohort median and searches the 0.02 ppm window resolved against the actual axis. The median limits a minority composition outlier's influence on its target; it does not turn whole-spectrum alignment into region-dependent warping or prove that every chosen shift is chemically correct. Use align=None (CLI: --align none) when a cohort should retain its referenced axes without mutual alignment.

The processing log records the reference strategy and applied window. A WINDOW SATURATED line means only that an unconstrained correlation maximum lay outside that window. Its absence is not a correctness signal: the harmful ST000049 cohort-mean shift was inside 0.02 ppm and produced no warning. A known small rigid-drift positive control and the public ST000049 regression receipt are both tested; larger, fractional, or region-dependent drift is not promised. The lower-level correlation_shift_align(reference=None) function retains its cohort-mean target for compatibility, while callers can pass an explicit target.

Reference and QC notes

The software chooses the reference from both pulse program and sample context. Aqueous biofluids use TSP/DSS at 0.000 ppm. Non-aqueous spectra with explicit TMS evidence use TMS at 0.000 ppm. CDCl3 lipid extracts without TMS evidence use residual CHCl3 at 7.260 ppm and exclude 7.24-7.28 ppm from the matrix. If pdata/1/procs or pdata/1/proc is present, NMRMetaProc uses Bruker OFFSET, SW_p, and SF to construct the initial ppm-axis geometry and records ppm_axis_source=bruker_procs in acquisition_parameters.csv; metadata presence alone is not reported as a reference lock. With referencing enabled, the selected peak is measured on either axis source. A robust measured peak can correct a processed axis, while an unverified processed axis is retained unchanged and explicitly reported as reference_locked=false with a REFERENCE UNVERIFIED warning. A failed acquisition-axis lock is likewise reported, with acceptance depending on the selected reference and context policy rather than an assumption of calibration. QC SNR is computed from the main metabolite window after excluding the active residual solvent/window, so a bad water peak cannot inflate the signal score. The residual-region score is the residual maximum divided by the maximum non-residual signal; values above 1.0 mean the residual is stronger than any non-residual signal. Moderate residual water in an excluded aqueous water window remains an exported diagnostic, while severe residual dominance becomes a warning. WARN samples are accepted into the matrix but carry review items; clean PASS, WARN, and FAIL are reported separately in the GUI and QC report.

Public benchmark rules are recorded in benchmarks/PUBLIC_BENCHMARKS.md and benchmarks/public_benchmark_manifest.json. Private or collaborator samples are not default benchmarks. A sample can be called source-published pass, treatable, or fail only when the exact label and source URL are recorded in the manifest.

A public Bruker 1D FID fixture from the MMCD/nmrglue example mirror is included in the test suite to validate raw FID reading, PULPROG alias dispatch, ppm-axis direction, and dominant solvent-region placement.

For a biological metabolomics benchmark, benchmarks/pepsnmr_human_serum/ provides a reproducible runner for the public Bioconductor PepsNMRData package. It downloads raw Bruker human-serum FIDs plus the package's published FinalSpectra_HS matrix, processes the raw FIDs with NMRMetaProc, and writes an auditable benchmark JSON. PepsNMRData does not provide human QC pass/fail labels, so the benchmark checks external discovery, pulse-program dispatch, finite matrix export, accepted-row agreement with the published 32-sample processed matrix, and transparent QC diagnostics rather than forcing artificial QC labels.

The 2026-07-26 scientific run identified and fixed a Bruker convention defect: GRPDLY=-1 means derive the digital-filter delay from DECIM/DSPFVS, not skip correction. Against Re(FinalSpectra_HS), the corrected 32-sample output reached median Pearson 0.9800 (minimum 0.9587), median cosine 0.9820 (minimum 0.9624), median absolute residual shift 0.0025 ppm, and 95th-percentile absolute residual shift 0.0040 ppm. These pass the criteria declared before the fix in the independent benchmark oracle.

All 32 linewidth estimates remain unreliable and are surfaced as WARN/indeterminate rather than silent clean passes. Because PepsNMRData publishes no sample-level pass/treatable/fail labels, this validates the declared spectral-agreement scope and transparent status handling only; it does not establish biological QC accuracy, linewidth calibration, or production readiness.

The independent runner in benchmarks/metabolomics_workbench/ extends the preprocessing evidence to two checksum-bound, public-domain-after-embargo datasets without relaxing the PepsNMRData criteria. The recorded v1.0.30 ST000049 run processed and exactly mapped 13/13 insect-extract, 950 MHz noesypr1d FIDs, reaching median Pearson 0.9568 and median cosine 0.9636 against 248 usable depositor bins. A later 0.02 ppm cohort-mean alignment default regressed Control_X to Pearson/cosine 0.819528/0.862030 without a saturation warning. The current pointwise-median pipeline reference passes the complete checksum-bound rerun at the same window: minimum Pearson/cosine 0.936941/0.945661, Control_X 0.956578/0.963597, and median/p95 residual shift 0.000/0.000 ppm. The reduced public receipt and full rerun instructions are in tests/data/st000049_alignment_receipt/.

The recorded v1.0.30 ST000306 run processed 113/113 human-plasma, 700 MHz instrument-declared cpmgpr1d FIDs and exactly mapped all 110 depositor rows, reaching median Pearson 0.9690, median cosine 0.9735, and median/p95 absolute shift 0.006/0.016 ppm against 142 usable bins.

Both studies identify repeated pooled material, and both the published and regenerated matrices preserve higher within-pool similarity than ordinary sample-pair similarity. This is evidence for technical repeatability only. Neither source publishes per-sample pass/treatable/fail labels, so these runs do not validate biological QC classification, universal thresholds, or production readiness. See the Workbench benchmark documentation for checksums, exclusions, commands, and source limitations.

To rerun the local automated public-evidence gate after the benchmark output exists:

python benchmarks/validate_public_release.py

The checksum-bound ST000020 urine benchmark independently re-integrates every deposited pdata/1/1r spectrum before comparing NMRMetaProc with the depositor matrix. The guard identifies one predeclared source-invalid row and then compares the remaining 107 rows symmetrically. Under the documented source-compatible urine workflow, it reaches median/minimum Pearson 0.99418/0.96385 and median/minimum cosine 0.99502/0.96562. Its pooled samples remain a repeatability proxy only: the source does not publish sample-level pass/treatable/fail labels.

Release 1.0.37 sets the first-order phase once per acquisition set-up. Run through the same benchmark scripts and criteria (median and per-sample Pearson and cosine at least 0.95 and 0.90, median and 95th-percentile shift at most 0.01 and 0.02 ppm), ST000049, ST000306 and ST000020 pass, with median/minimum Pearson 0.991/0.975, 0.987/0.957 and 0.996/0.954. PepsNMRData does not: its minimum Pearson is 0.822 and minimum cosine 0.833, with medians of 0.979 and 0.981. The PepsNMRData matrix was phased by PepsNMR's own zero-order rule, which in its 12 lowest-agreement samples is further from absorption on the formate line than this release's phase step, in the opposite direction. Counts and reasons are in docs/phase_correction_1.0.37_acceptance.md.

Citing and Provenance

If you use nmrmetaproc in your research, please cite:

Omage, F. B., Omage, T. B., & Tasic, L. (2026). nmrmetaproc: NMR Metabolomics Spectral Processor (Version 1.0.37).
Zenodo. https://doi.org/10.5281/zenodo.19194107

The canonical public provenance records are the GitHub repository, annotated release tags, GitHub Releases, CITATION.cff, AUTHORS.md, NOTICE, PROVENANCE.md, and the Zenodo DOI. The MIT License permits reuse and redistribution only with the required copyright and permission notices retained. Scientific use should preserve authorship and cite the software DOI.

The PQN normalization method:

Dieterle, F., Ross, A., Schlotterbeck, G., & Senn, H. (2006). Probabilistic quotient normalization as robust method to account for dilution of complex biological mixtures. Analytical Chemistry, 78(13), 4281-4290. https://doi.org/10.1021/ac051632c

Development

git clone https://github.com/omagebright/nmrmetaproc.git
cd nmrmetaproc
pip install -e ".[dev]"
pytest tests/ -v

License

MIT License. See LICENSE for details. Redistribution and derivative work must retain the copyright and permission notices. See NOTICE and PROVENANCE.md for authorship and citation records.