Skip to main content

spectrum-reader

Small Rust library that reads single-pixel spectra from hyperspectral HDF5 files without libhdf5. Designed for use in the SIDS.

Targets

  • Linux x86_64 (Debian trixie): the production build. It is built on Debian trixie and its binary is the one shipped to the SIDS.
  • macOS arm64: native library for local development and Python wheels (macOS 11+).
  • Windows x86_64: native library and Python wheels, built with the MSVC target.

The Python Linux wheels target x86_64 with glibc 2.34 or newer, including Debian trixie.

C API

The declarations below are those of include/spectrum_reader.h.

uint32_t    sr_abi_version(void);
const char *sr_last_error_message(void);
int32_t     sr_open(const char *path, SrFile **out);
void        sr_close(SrFile *file);
int32_t     sr_shape(const SrFile *file, uint64_t *out);
int32_t     sr_wavelengths(const SrFile *file, double *out, size_t len);
int32_t     sr_read_pixel(const SrFile *file, uint64_t row, uint64_t col, float *out, size_t len);
  • Functions returning int32_t return 0 on success, otherwise one of: 1 internal error (including a caught panic), 2 invalid argument (null pointer, wrong buffer length), 3 I/O error, 4 unsupported or corrupt file, 5 pixel out of range.
  • After a failed call, sr_last_error_message() returns a description. The message is kept per thread, is replaced by the next call on the same thread, and the pointer is only valid until then: copy it right away.
  • path is a NUL-terminated byte string: filesystem bytes on Unix, UTF-8 on Windows. sr_open sets *out to NULL on failure.
  • sr_shape writes three values: rows, cols, bands. row and col are the first two indices of the spectra dataset.
  • The caller owns every output buffer. len must equal the number of bands exactly, otherwise the call fails and the buffer is left untouched.
  • A handle can be shared between threads for sr_shape, sr_wavelengths and sr_read_pixel. sr_close must not run while another call uses the handle, and a closed handle must not be used again. sr_close(NULL) does nothing.
  • Only little-endian float32 spectra (3 dimensions) and float64 Wavelength (1 dimension) with contiguous storage are supported. Anything else is rejected at sr_open.

Development

rustup is required. The toolchain is pinned in rust-toolchain.toml and rustup installs it automatically the first time cargo runs.

make test
make build
make smoke
make lint

make smoke compiles tests/smoke.c against the built library and include/spectrum_reader.h, then runs it.

The C header is generated and committed. After changing the exported functions, run make header; make check-header fails if the committed header is out of date. This needs cbindgen 0.29.4:

cargo install cbindgen --version 0.29.4 --locked

The release library is target/release/libspectrum_reader.so (.dylib on macOS).

Building on macOS (arm64)

Install the Xcode command-line tools (xcode-select --install) and rustup, then from the repository root:

make build

The library is target/release/libspectrum_reader.dylib. Point the SIDS wrapper at it for local development, together with include/spectrum_reader.h.

Building the Linux library from macOS

An alternative to building on Debian trixie. zig is used as the linker, and the glibc version the library requires is pinned in the target name (.2.34; trixie ships 2.41, so anything up to that works).

rustup target add x86_64-unknown-linux-gnu
brew install zig
cargo install cargo-zigbuild --version 0.24.0 --locked
make build-linux

The library is target/x86_64-unknown-linux-gnu/release/libspectrum_reader.so. It cannot run on the Mac, so make smoke does not test it: make check tests the macOS build of the same code.

Python bindings and wheels

The spectrum-reader Python package bundles the native library and uses CFFI to call the C API. It supports Python 3.10+ and needs neither Rust nor libhdf5 on the machine where a wheel is installed. Its only Python runtime dependency is cffi, which pip installs automatically.

from spectrum_reader import SpectrumFile

with SpectrumFile("image.h5") as spectrum:
    rows, cols, bands = spectrum.shape
    wavelengths = spectrum.wavelengths
    values = spectrum.read_pixel(row=0, col=0)

Wavelengths and pixel values are returned as lists of Python floats. Coordinates are zero-based. Paths can be strings, bytes, or pathlib.Path objects. Use the context manager or call close() explicitly; repeated closes are harmless. Calls on the same object, including close(), are synchronized.

Missing or unreadable files raise OSError; unsupported or corrupt files raise spectrum_reader.FormatError (a ValueError); out-of-range pixels raise IndexError. Using a closed file raises ValueError. Internal native errors raise spectrum_reader.SpectrumError (a RuntimeError).

Build all three wheels from macOS

Install uv and the Rust toolchain, then prepare the cross-compilation tools:

rustup target add aarch64-apple-darwin x86_64-unknown-linux-gnu x86_64-pc-windows-msvc
brew install zig llvm lld
export PATH="$(brew --prefix llvm)/bin:$(brew --prefix lld)/bin:$PATH"
make build-wheels

The build targets run Maturin 1.15.0 and CFFI through uv. Linux builds use Zig; Windows builds use Maturin's integrated xwin support, which downloads the Microsoft CRT and Windows SDK on the first build (under the xwin license terms). Building the Darwin wheel requires a macOS host and the Xcode command-line tools.

The wheels are written to dist/ (override with WHEEL_OUT=path):

Make target Wheel platform tag
build-wheel-darwin-arm64 macosx_11_0_arm64
build-wheel-linux-amd64 manylinux_2_34_x86_64
build-wheel-windows-amd64 win_amd64

Each wheel has the py3-none Python/ABI tags, so a separate build for each Python version is unnecessary. Run an individual target to build just that platform, or make build-wheel for the current host. On Windows, run Make from a shell with the MSVC build tools available; native builds do not use xwin.

Install the wheel matching your platform, for example on an Apple Silicon Mac:

python -m pip install dist/spectrum_reader-0.1.0-py3-none-macosx_11_0_arm64.whl

Test the Python package

make test-python

This builds and installs the package into an isolated uv environment and runs the Python integration tests against the HDF5 fixtures. To test a particular built wheel on its target platform:

uv run --no-project --with /path/to/spectrum_reader.whl python -m unittest discover -s tests/python -v

Use the wheel's actual filename. Cross-compiled wheels must be tested on their respective operating systems; the local tests exercise only the host platform.

Updating the library in the SIDS

Build from a clean working tree at the commit being shipped, either on Debian trixie or from macOS (see above).

  1. Run all the checks; they must pass:
make check

It runs clippy (warnings are errors), the tests, the header staleness check and the C smoke test.

  1. If built on Debian trixie, check the glibc requirement of the library; the highest version printed must not exceed trixie's (2.41). A library from make build-linux is pinned to 2.34 already:
objdump -T target/release/libspectrum_reader.so | grep -o 'GLIBC_[0-9.]*' | sort -uV | tail -1
  1. Copy these two files into the SIDS repository:

    • target/release/libspectrum_reader.so (target/x86_64-unknown-linux-gnu/release/libspectrum_reader.so if built with make build-linux)
    • include/spectrum_reader.h
  2. If the exported functions or their behavior changed incompatibly, bump ABI_VERSION in src/lib.rs first, run make header, and update the version the SIDS wrapper expects from sr_abi_version().

License

CeCILL-B. See LICENSE.txt.

Metadata

Release files for spectrum-reader-certic 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for spectrum-reader-certic 0.1.0
File Interpreter ABI Platform Actions
spectrum_reader_certic-0.1.0-py3-none-win_amd64.whl Python 3 none Windows x86-64 Details for spectrum_reader_certic-0.1.0-py3-none-win_amd64.whl
spectrum_reader_certic-0.1.0-py3-none-manylinux_2_34_x86_64.whl Python 3 none Linux glibc 2.34+ x86-64 Details for spectrum_reader_certic-0.1.0-py3-none-manylinux_2_34_x86_64.whl
spectrum_reader_certic-0.1.0-py3-none-macosx_11_0_arm64.whl Python 3 none macOS 11.0+ ARM64 Details for spectrum_reader_certic-0.1.0-py3-none-macosx_11_0_arm64.whl

Total release size: 426.3 kB

Release files / spectrum_reader_certic-0.1.0-py3-none-win_amd64.whl

Download URL spectrum_reader_certic-0.1.0-py3-none-win_amd64.whl
Size 85.9 kB
Tags Python 3 Windows x86-64
SHA-256 checksum
How to use checksums
2501dddcab658868fd4275cd292a1801626b48fd56b570c25a8fe80761a311bf
BLAKE2b-256 checksum
How to use checksums
b75ec7ba751edeffa8800c7bb7df64afde15da85f82e0ed29673ed6651f001d2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via maturin/1.15.0

Release files / spectrum_reader_certic-0.1.0-py3-none-manylinux_2_34_x86_64.whl

Download URL spectrum_reader_certic-0.1.0-py3-none-manylinux_2_34_x86_64.whl
Size 180.6 kB
Tags Linux glibc 2.34+ x86-64 Python 3
SHA-256 checksum
How to use checksums
5a9b6437980cd96d4e236f81b9fdc0cc9dcf0e942cc48ab81731253c3e038f95
BLAKE2b-256 checksum
How to use checksums
74ec5fbdd017680de90fe10970bfd73aa05ef0f1f473f628af4e56d6e6e808e6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via maturin/1.15.0

Release files / spectrum_reader_certic-0.1.0-py3-none-macosx_11_0_arm64.whl

Download URL spectrum_reader_certic-0.1.0-py3-none-macosx_11_0_arm64.whl
Size 159.8 kB
Tags Python 3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
a135353c900a0df6b101d8297de5f2cc8b78ebb2865b78f36e2a390cb4f426df
BLAKE2b-256 checksum
How to use checksums
cb836c1595ec1c974d70e1c2093d4b6cfb2d1d58b7d34b14ca8aef0f6c3c39de
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via maturin/1.15.0

Security for spectrum-reader-certic 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

3 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page