hyperspec-py 0.7.0

Fast hyperspectral data processing for remote sensing and ML pipelines

hyperspec

Fast hyperspectral data processing for remote sensing and ML pipelines. Rust core with Python bindings.

Built by Hyperstate — foundation models for Earth intelligence.

Why hyperspec?

Hyperspectral imagery produces hundreds of bands per pixel, orders of magnitude more than RGB or multispectral. Most existing tools are slow, have heavy system dependencies, and were not designed for modern ML training pipelines.

hyperspec provides:

• Fast Rust implementations of standard hyperspectral algorithms
• A SpectralCube abstraction that keeps data, wavelengths, and metadata together
• Zarr V3 storage with chunking optimized for ML dataloaders
• Python bindings with zero-copy NumPy interop

Install

Python:

pip install hyperspec-py
# or
uv add hyperspec-py

Rust:

cargo add hyperspec

or

[dependencies]
hyperspec = "0.7"

Quickstart

import numpy as np
from hyperspec import (
    SpectralCube, read_envi, write_zarr, read_zarr_window, zarr_cube_shape,
    sam, continuum_removal, pca, mnf_denoise,
    band_stats, normalize_zscore, savitzky_golay, derivative,
)

# --- I/O ---

# Read ENVI format (AVIRIS-NG, etc.)
cube = read_envi("scene.hdr")

# Process
cr_cube = continuum_removal(cube)
denoised = mnf_denoise(cube, n_components=20)

# Write to Zarr with ML-friendly chunks
write_zarr(cube, "chips.zarr", chunk_shape=(cube.bands, 256, 256))

# Read a single training tile without loading the full cube
tile = read_zarr_window(
    "chips.zarr",
    bands=(0, cube.bands),
    rows=(0, 256),
    cols=(0, 256),
)

# Query shape without loading data
bands, lines, samples = zarr_cube_shape("chips.zarr")

# --- Preprocessing ---

# Per-band statistics
stats = band_stats(cube)        # .mean, .std, .min, .max, .valid_count

# Normalize for ML training
normed = normalize_zscore(cube)

# Smooth noisy spectra, then compute derivatives
smooth = savitzky_golay(cube, window=7, polyorder=2)
d1 = derivative(smooth, order=1)

# --- Spectral analysis ---

# Compare every pixel to a reference spectrum
reference = cube.spectrum(50, 50)
angles = sam(cube, reference)

# Dimensionality reduction
pca_result = pca(cube, n_components=10)

SpectralCube

Category	Methods
Dimensions	bands, height, width, shape
Data access	data(), wavelengths(), fwhm(), nodata
Pixel access	spectrum(row, col)
Band access	band(index), band_at(nm), band_nearest(nm)
Wavelength lookup	wavelength(index), wavelength_index(nm), nearest_band_index(nm)
Statistics	mean_spectrum()
Subsetting	sel_bands([i, j]), sel_wavelengths(min, max)

I/O

Read

Format	Function
ENVI	read_envi(path)
Zarr V3	read_zarr(path)
Zarr V3	read_zarr_with_options(path, ...)
Zarr V3	read_zarr_window(path, bands, rows, cols)

Write

Format	Function
ENVI	write_envi(cube, path)
Zarr V3	write_zarr(cube, path)

Utilities

Format	Function
Zarr V3	zarr_cube_shape(path)

Algorithms

Spectral similarity

Operation	Function
Spectral Angle Mapper	sam(cube, reference)

Spectral preprocessing

Operation	Function
Continuum removal	continuum_removal(cube)
Savitzky-Golay smoothing	savitzky_golay(cube, window, polyorder)
Derivative spectra	derivative(cube, order)
Min-max normalization	normalize_minmax(cube)
Z-score normalization	normalize_zscore(cube)

Statistics

Operation	Function
Per-band stats	band_stats(cube) → BandStats
Covariance matrix	covariance(cube)
Correlation matrix	correlation(cube)

Spectral indices

Operation	Function
Normalized difference	normalized_difference(cube, a, b)
Band ratio	band_ratio(cube, a, b)
NDVI	ndvi(cube, nir, red)

Dimensionality reduction

Operation	Function
PCA	pca(cube, n_components)
MNF	mnf(cube, n_components)
MNF denoise	mnf_denoise(cube, n_components)

Spectral utilities

Operation	Function
Resample	resample(cube, target_wl, method)

Architecture

crates/hyperspec/           # Pure Rust library → crates.io
└── src/
    ├── cube.rs             # SpectralCube type
    ├── error.rs            # HyperspecError
    ├── io/
    │   ├── envi.rs         # ENVI read/write (BSQ, BIL, BIP)
    │   └── zarr.rs         # Zarr V3 read/write/window
    └── algorithms/
        ├── sam.rs
        ├── continuum.rs
        ├── indices.rs
        ├── pca.rs
        ├── mnf.rs
        ├── resample.rs
        ├── stats.rs
        ├── normalize.rs
        ├── derivative.rs
        └── smooth.rs

pyo3-hyperspec/             # PyO3 bindings → PyPI: hyperspec-py
├── src/lib.rs
└── python/hyperspec/
    └── __init__.py

• Rust 2024 edition, using ndarray, rayon, zarrs, thiserror
• Minimal system dependencies — no LAPACK or GDAL
• Python 3.12+, NumPy bindings via PyO3

Roadmap

• Cloud-native Zarr stores (Azure, GCS, S3)
• More algorithms and utilities

Development

uv venv && source .venv/bin/activate
uv pip install maturin pytest
cargo test
maturin develop
pytest

License

Apache-2.0