astrospecflow 0.1.0

Tool-native spectral pipeline runtime with stage-aware profiles and artifact tracing.

AstroSpecFlow

AstroSpecFlow is a Python package for running a LAMOST-style one-dimensional spectral redshift reduction workflow. It provides a command-line interface, Python APIs, optional CUDA acceleration hooks, and an optional MySQL publication sink for deployment environments that store reduced spectra in a database.

The package is designed for site deployment. It does not ship private FITS files, template libraries, database credentials, output artifacts, or deployment-specific paths. Operators provide those resources through command line flags, environment variables, or a local config file that is not committed to git.

Install

From a source checkout:

python3 -m pip install -e '.[dev]'

After installation:

python3 -c "import astrospecflow; print(astrospecflow.__version__)"
astrospecflow-reduce1d --help

Importing astrospecflow is intentionally lightweight. It must not compile or load CUDA extensions, read FITS files, search template directories, or connect to MySQL. Those resources are checked only when a user explicitly runs the pipeline.

Configuration

Copy the example config and edit it for the local deployment:

cp configs/astrospecflow.example.toml astrospecflow.local.toml

Do not commit astrospecflow.local.toml or .env; both are ignored by git.

Configuration precedence is:

1. CLI flags
2. ASTROSPECFLOW_* environment variables
3. --config / ASTROSPECFLOW_CONFIG local TOML file
4. documented package defaults

The MySQL host, port, user, password, database, table, SSL settings, FITS root, template root, output directory, CUDA device, and batch size are deployment settings. They must be provided by the operator and are not embedded in the package.

Quickstart

Run a no-database smoke test:

astrospecflow-reduce1d /path/to/spec-file-1.fits /path/to/spec-file-2.fits \
  --device cpu \
  --batch-size 2 \
  --artifact /path/to/local-output/smoke-no-mysql.json

Run with a local config file:

astrospecflow-reduce1d --config astrospecflow.local.toml \
  /path/to/spec-file-1.fits \
  --artifact /path/to/local-output/config-smoke.json

Run with MySQL output:

astrospecflow-reduce1d --config astrospecflow.local.toml \
  /path/to/spec-file-1.fits /path/to/spec-file-2.fits \
  --write-mysql \
  --artifact /path/to/local-output/mysql-smoke.json

If required MySQL fields are missing, the CLI fails before the long reduction starts and reports which fields must be configured.

CUDA Policy

The 0.1.0 release line is a Python orchestration package. CUDA acceleration is optional and depends on a local extension build that matches the deployment host's CUDA, PyTorch, and GPU architecture. A missing CUDA extension should not break package import, documentation builds, or CPU smoke tests.

Validation

Local release checks:

python3 scripts/release/audit_release_surface.py
PYTHONPATH=src pytest -q
python -m sphinx -b html docs docs/_build/html
python -m build
python -m twine check --strict dist/*
bash scripts/release/validate_pypi_candidate.sh

A wheel is the built package file that pip normally installs. An sdist is the source distribution archive. twine check validates package metadata and README rendering before upload to TestPyPI or PyPI.

Deployment promotion can add a fixed regression set. In this repository, release32 means a site-owned validation set of 32 FITS files, typically about 8000 spectra, used to compare end-to-end throughput and MySQL publication fields. It is not required for ordinary installation or quickstart usage.

PyPI

Build locally:

python -m build
python -m twine check --strict dist/*

TestPyPI verification, when credentials are available:

python -m twine upload --repository testpypi dist/*
python3 -m venv /tmp/astrospecflow-testpypi
/tmp/astrospecflow-testpypi/bin/python -m pip install \
  --index-url https://test.pypi.org/simple/ --no-deps astrospecflow

Real PyPI publication requires a PyPI API token or GitHub Trusted Publishing. After publication, users can install with:

python3 -m pip install astrospecflow

Documentation

Build docs locally:

python -m sphinx -b html docs docs/_build/html

Start with docs/index.rst, docs/install.rst, docs/quickstart.rst, docs/configuration.rst, docs/resource_contract.rst, docs/cuda_extension.rst, docs/mysql_sink.rst, docs/validation.rst, and docs/troubleshooting.rst.