raman-data 0.0.3

A unified Python package for accessing various Raman spectroscopy datasets.

Raman-Data: A Unified Python Library for Raman Spectroscopy Datasets

This project aims to create a unified Python package for accessing various Raman spectroscopy datasets. The goal is to provide a simple and consistent API to load data from different sources like Kaggle, Hugging Face, GitHub, and Zenodo. This will be beneficial for the Raman spectroscopy community, enabling easier evaluation of models, such as foundation models for Raman spectroscopy.

✨ Features

• A single, easy-to-use Python package (planned for PyPI).
• Automatic downloading and caching of datasets from their original sources.
• A unified data format for all datasets.
• A simple function to list available datasets, with filtering options.

🚀 Getting Started

The basic interface for the package is defined in raman_data/__init__.py. Here's a preview of how it will work:

from raman_data import raman_data
# To specify a task type import this enum as well
from raman_data import TASK_TYPE

# List all available datasets
print(raman_data())

# List only classification datasets
print(raman_data(task_type=TASK_TYPE.Classification))

# Load a dataset
dataset = raman_data(name="codina/diabetes/AGEs")

# Access the data, targets, and metadata
X = dataset.data
y = dataset.target
metadata = dataset.metadata

print(X.shape)
print(y.shape)
print(metadata)

For more detailed examples see Demo Notebook.

📚 Available Datasets

Here is the list of datasets that are currently included in the package:

Kaggle

• Diabetes Spectroscopy
• Liquid Chromatography

Hugging Face

• Substrate Mix Raman
• Ecoli Fermentation
• Fuel Spectra Benchtop

Zenodo

• Hyperspectral Unmixing
• Mutant Wheat Lines
• Surface Enhanced Spectroscopy for quantitative analysis

🎯 Milestones

• View Datasets
• Software architecture with dummy data
• Software tests
• Integration of Kaggle
• Integration of Huggingface
• Integration of Github
• Integration of Zenodo
• Integration of other datasets
• Finalize Package
  • Documentation
  • Publish to PyPi

🔮 For Later (Future Datasets)

Kaggle

• Cancer Cells SERS Spectra (requires authentification)

GitHub

• Raman Spectra Data
• Raman spectra of pathogenic bacteria (more info on this GitHub page)
• High-throughput molecular imaging
• spectrai raman spectra

Zenodo

• Quantitative volumetric Raman imaging

Other Sources

• Spectra of illicit adulterants
• Raman Spectrum Matching with Contrastive Representation Learning
• Raman spectra of chemical compounds
• Inline Raman Spectroscopy and Indirect Hard Modeling
• The Effect of Sulfate Electrolytes on the Liquid-Liquid Equilibrium
• In-line Monitoring of Microgel Synthesis (weird format)
• N-isopropylacrylamide Microgel Synthesis
• Nonlinear Manifold Learning Determines Microgel Size from Raman Spectroscopy
• NASA AHEAD
• RRUFF