RSRF

RSRF is a repository-backed Python toolkit and curated data repository for canonical optical sensor spectral response definitions.

It keeps the pieces that usually drift apart in one place:

• a read API for sampled response curves and metadata-only band specs
• a CLI for inspection, validation, and registry operations
• manifest-driven ingest and provenance tracking for official upstream artifacts
• checked-in canonical outputs, QA assets, and documentation visualizations

The distribution name is RSRF. The import package is rsrf.

Documentation: https://marcyin.github.io/spectral_response/

Interactive visualization: https://marcyin.github.io/spectral_response/visualizations/

What Lives In This Repository

• canonical sampled curves under data/canonical/sampled_curve/
• canonical metadata-only band specs under data/canonical/band_spec/
• parquet registries under data/registry/
• raw, extracted, and manifest-tracked source artifacts under sources/
• Python package code under src/rsrf/
• MkDocs content and browser visualization assets under docs/

The current repository includes both multispectral and hyperspectral coverage, including Sentinel-2, Sentinel-3 OLCI and SLSTR, Landsat, MODIS, VIIRS, ASTER, PlanetScope, SkySat, PROBA-V, PRISMA, EMIT, EnMAP, PACE OCI, Pleiades, FORMOSAT-5, Amazonia-1, and CBERS-4A families.

Installation

For repository development:

python3 -m pip install -e ".[dev]"

For a minimal install from a checkout:

python3 -m pip install .

Optional extras are split by workflow:

• .[ingest] for HDF5, NetCDF, and xarray-based source parsers
• .[qa] for validation plots
• .[docs] for MkDocs site builds
• .[release] for packaging checks
• .[lint] for ruff-based code quality checks

The .[dev] extra composes all of the above.

Repository-Backed Data Access

The Python package still ships code separately from the full repository snapshot, but installed-package workflows no longer need a manual checkout. When RSRF is used outside a repository checkout and no explicit root is provided, it downloads the matching GitHub release snapshot into a local cache and reads canonical data from there.

Root resolution now follows this order:

• pass root= or --root /path/to/custom/root
• set RSRF_ROOT=/path/to/custom/root
• run from inside a repository checkout
• fall back to the cached GitHub release snapshot for the installed version

Quick Start

From a repository checkout:

python3 -m rsrf --help
export RSRF_ROOT="$PWD"
rsrf show-layout
rsrf list-sensors
rsrf list-bands sentinel-2c_msi --variant band_average
rsrf show-response sentinel-2c_msi B03 --variant band_average
rsrf show-metadata prisma_hsi --variant metadata_band_spec
rsrf validate-sensor sentinel-2c_msi --variant band_average

From Python:

from pathlib import Path

from rsrf import get_metadata, list_sensors, load_response_definition

root = Path(".")
sensors = list_sensors(root=root)
metadata = get_metadata("sentinel-2c_msi", "band_average", root=root)
response = load_response_definition("sentinel-2c_msi", "B03", "band_average", root=root)

From an installed package outside a checkout, the same calls work without setting root; the first read will populate the local cache from the matching GitHub release.

load_response_definition() returns either a SampledCurve or a BandSpec, depending on the representation variant.

For runtime custom inputs, use coerce_response_definition() with either sampled points (wavelength_nm + response, or wavelength + relative_spectral_response), a band spec (center_wavelength_nm + fwhm_nm), or a zero-argument callable returning one of those forms. Downstream helpers realize center+FWHM inputs as Gaussian curves when they need sampled responses.

For whole-sensor interchange, use get_sensor_definition() for registry sensors and coerce_sensor_definition() or load_sensor_definition() for custom payloads. These APIs normalize canonical and user-supplied sensors into the same versioned SensorDefinition contract with per-band response_definition payloads and namespaced extensions.

Manifest And Registry Workflows

Manifest-aware commands accept either explicit paths or checked-in manifest filenames from the manifest library. From the repository root, these work without the full manifest path:

rsrf validate-manifest rsrf_source_manifest_sentinel2c_v2.json
rsrf show-registry-rows rsrf_source_manifest_prisma_hsi_v2.json
rsrf register-manifest rsrf_source_manifest_prisma_hsi_v2.json

Planning catalog support is exposed separately:

rsrf list-planned-sensors
rsrf register-planned-sensors

Checked-in manifests live under sources/manifests/official/, planning catalogs under sources/manifests/planning/, and templates under sources/manifests/templates/.

Data Model

RSRF currently works with two canonical representation types:

• sampled_curve: full spectral response samples stored in curves.parquet
• band_spec: metadata-only band definitions stored in band_specs.parquet

Each sensor representation also carries a metadata.json sidecar, and sampled-curve variants can include trusted overlay references at:

sources/extracted/<sensor_unit_id>/<representation_variant>/overlay_reference.csv

When only metadata-level band specs are available, RSRF can realize approximate sampled curves for QA and visualization workflows.

Repository Structure

.
|-- data/
|   |-- canonical/
|   |-- common_grid/
|   |-- realized/
|   `-- registry/
|-- docs/
|-- plans/
|-- scripts/
|   |-- build/
|   |-- ingest/
|   `-- validate/
|-- sources/
|   |-- extracted/
|   |-- manifests/
|   |   |-- official/
|   |   |-- planning/
|   |   `-- templates/
|   `-- raw/
|-- src/rsrf/
|   |-- commands/
|   `-- parsers/
`-- tests/

Useful code entry points:

• src/rsrf/api.py for read-side access to canonical sensor definitions
• src/rsrf/commands/ for CLI parser, dispatch, and output helpers
• src/rsrf/ingest.py for canonical artifact writing and registry updates
• src/rsrf/manifests.py for manifest library lookup and path resolution
• src/rsrf/planning.py for registry-first planning catalog support
• src/rsrf/qa.py for validation reports and plot export
• src/rsrf/registry.py for repository layout and parquet registry helpers
• src/rsrf/visualization.py and src/rsrf/docs_site.py for docs visualization asset export and site preparation

Documentation

Project documentation is built with MkDocs and published at https://marcyin.github.io/spectral_response/.

Key entry points:

• getting started: docs/getting-started.md
• Python API reference: docs/python-api.md
• interactive visualizations: docs/visualizations.md
• CLI reference: docs/cli.md
• data model: docs/data-model.md
• repository layout: docs/repository-layout.md
• development guide: docs/development.md
• release guide: docs/releasing.md

To refresh the generated docs site files and versioned visualization bundles:

python3 scripts/build/prepare_docs_site.py --root .

This refreshes docs/assets/visualization/, renders docs/visualizations.md from its template, and syncs the versioned JS and CSS bundles referenced by mkdocs.yml.

Development

Run the local verification stack with:

ruff check src/ tests/ scripts/
ruff format --check src/ tests/ scripts/
python3 -m unittest discover -s tests/unit
python3 -m unittest discover -s tests/regression
python3 scripts/build/prepare_docs_site.py --root .
python3 -m build
python3 -m twine check dist/*
mkdocs build --strict

Project Status

Active design and implementation history lives under plans/, including:

• plans/rsrf_repo_plan_v3_hyperspectral_support.md
• plans/rsrf_implementation_plan.md