pyquetms

Memory-efficient mzML to Parquet converter for mass spectrometry files.

Overview

pyquetms provides streaming conversion of mzML files to Parquet format with minimal memory usage, making it suitable for processing large mass spectrometry datasets without running out of memory. This project was originally developed as a side project inspired by GSoC 25' with OpenMS, with the goal of providing a simple CLI for converting .mzML to .parquet files, which is especially important in big data projects (e.g., machine learning).

Installation

From PyPI

pip install pyquetms

From source

git clone https://github.com/Avni2000/pyquetms.git
cd pyquetms
pip install .

Development installation

git clone https://github.com/Avni2000/pyquetms.git
cd pyquetms
pip install -e ".[dev]"

Usage

CLI

Basic conversion:

pyquetms input.mzML

or

pyquetms ~/Downloads/input.mzML

Specify output file (defaults to working directory):

pyquetms input.mzML -o output.parquet

Customize batch size and compression. I recommend :

pyquetms input.mzML --batch-size 5000 --compression gzip

Get file information without converting:

pyquetms input.mzML --info

Output Format

The converted Parquet files contain the following columns:

Depending on the type of mzml file, we have slightly different columns. Some columns may be blank, which is perfectly okay! It doesn't mean your mzml is wrong. The main expected values are time, m/z, and intensity

Contributions

It's quite a small project, feel free to make a PR or open an issue!