INSTALLATION

The project is managed with uv.

To install the hfcviewer command, run:

uv tool install .

To use the HQI client in your own project, add it as a dependency, optionally with the pandas and/or astropy extras:

uv add "pyhfc[pandas,astropy]"

or, with pip:

pip install "pyhfc[pandas,astropy]"

To use the development version, install it from the develop branch:

uv add "pyhfc @ git+https://github.com/HELIO-HFC/PyHFC.git@develop"

To set up a development environment, run:

uv sync

VIEWER

To run the hfcViewer, enter:

uv run hfcviewer

or, equivalently:

uv run python -m pyhfc.hfcviewer

By default, the viewer displays the most recent SDO/AIA observation at the current date and time, with the active regions and the coronal holes detected on it (note that the HFC has no coronal holes after 2013-10-13). The Previous and Next buttons browse the observations of the selected data set, and the Latest button goes back to its most recent observation.

For instance, to display the Meudon H-alpha observation nearest to a date, with the filaments and the prominences:

uv run hfcviewer -d 2012-06-01T12:00:00 -o Meudon -i Spectroheliograph -w Halpha -f fi,pr

The features to display are given with -f (ar: active regions, ch: coronal holes, sp: sunspots, rs: radio sources, pr: prominences, fi: filaments; -f "" for none).

By default, the contours of a feature are the ones detected at the date closest to the date t of the displayed observation. A time range dt can be set for each feature in Options > Set time ranges: the contours detected in [t - dt, t + dt] are then all displayed on the same image, e.g. to follow the evolution of the active regions. The contours are then colored by tracking by default (a feature keeps its color from one observation to the other); Contour colors allows to color them by feature again.

The status bar shows two lights telling whether the HQI service (observations and features) and the server of the quicklook images (FTP) are reachable (green) or not (red). They are checked every minute, in the background; the interval can be changed in the Settings menu (Server checks). The version of pyhfc, at the right of the status bar, is a link to its PyPI page, and the LIRA logo a link to the LIRA website. Help > HFC database content opens the web page giving the content of the HFC database (observations and features available).

The quicklook images, and the data returned by the HQI (observations and features), are kept in a disk cache, so that they are displayed faster the next times (about 7 times faster for SDO/AIA images). The data of the last 7 days are cached for one hour only, as the HFC may still complete them, and the most recent observation (Latest button) is always queried. 50 SDO/AIA images with their active regions take about 50 MB. The cache is stored in the user cache directory (~/Library/Caches/pyhfc on macOS, ~/.cache/pyhfc on Linux; set the PYHFC_CACHE_DIR environment variable to use another directory). Its maximum size is 500 MB by default (about 500 SDO/AIA images): it can be changed, and the cache emptied, in the Settings menu. When the cache is full, the viewer says it, and the new images and data are no longer cached.

To display help, run:

uv run hfcviewer --help

Note: some old uv-managed Python builds (e.g. 3.12.12, 3.14.2) cannot load Tcl/Tk (“Can’t find a usable init.tcl”); install a newer patch release with uv python install.

HQI CLIENT

The HFC is queried through the HELIO Query Interface (HQI), a SOAP web service returning VOTables. pyhfc.api.HQIClient wraps all its methods:

and adds a few helpers:

• nearest(table, date), previous(table, date), next(table, date): the row observed nearest to, before or after a date;

• features(feature, start, end): the features (“active regions”, “sunspots”, “filaments”, “prominences”, “coronal holes”, “type III”…) observed between two dates;

• quicklook(row): the quicklook image (PIL) of an observation.

The main tables are VIEW_OBS_HQI (observations), VIEW_PP_HQI (pre-processed observations), VIEW_AR_HQI (active regions), VIEW_SP_HQI (sunspots), VIEW_CH_HQI (coronal holes), VIEW_FIL_HQI (filaments), VIEW_PRO_HQI (prominences), VIEW_RS_HQI (radio sources), VIEW_T2_HQI and VIEW_T3_HQI (type II and III radio bursts), and FRC_INFO (feature recognition codes).

from pyhfc.api import HQIClient

hqi = HQIClient()
print(hqi.tables())
print(hqi.fields("VIEW_AR_HQI"))

obs = hqi.nearest(
    "VIEW_OBS_HQI",
    "2012-06-01T12:00:00",
    where={"OBSERVAT": "SDO", "INSTRUME": "HMI", "TELESCOP": "CONTINUUM"},
)
print(obs["DATE_OBS"], obs["NAXIS1"], obs["CDELT1"], obs["QCLK_FNAME"])
hqi.quicklook(obs).save("hmi.png")

ars = hqi.select(
    "VIEW_AR_HQI",
    what="DATE_OBS, NOAA_NUMBER, FEAT_HG_LAT_DEG, FEAT_HG_LONG_DEG, FEAT_AREA_DEG2",
    where={
        "OBSERVAT": "SDO",
        "NOAA_NUMBER": (1, None),
        "DATE_OBS": ("2012-06-01", "2012-06-02"),
    },
    order_by="DATE_OBS, NOAA_NUMBER",
)
for row in ars:
    print(row["DATE_OBS"], row["NOAA_NUMBER"], row["FEAT_HG_LAT_DEG"], row["FEAT_HG_LONG_DEG"])

spots = hqi.features(
    "sunspots", "2012-06-01T00:00:00", "2012-06-01T12:00:00", where={"OBSERVAT": "SDO"}
)
df = spots.to_pandas()
print(df[["DATE_OBS", "FEAT_X_ARCSEC", "FEAT_Y_ARCSEC", "FEAT_AREA_DEG2"]])

big = hqi.query(
    "VIEW_AR_HQI",
    "2012-06-01T00:00:00",
    "2012-06-02T00:00:00",
    where={"OBSERVAT": "SDO", "FEAT_AREA_DEG2": (50, None)},
)
print({row["NOAA_NUMBER"] for row in big})

count = hqi.select(
    "VIEW_T3_HQI", what="COUNT(*) AS N", where={"DATE_OBS": ("2010-01-01", "2011-01-01")}
)
print(count.first()["N"])

filament = hqi.features(
    "filaments", "2011-06-01", "2011-06-02", what="TRACK_ID", where="TRACK_ID > 0", limit=1
).first()
track = hqi.select(
    "VIEW_FIL_HQI",
    what="DATE_OBS, FEAT_HG_LAT_DEG, FEAT_HG_LONG_DEG, SKE_LENGTH_DEG",
    where={"TRACK_ID": filament["TRACK_ID"]},
    order_by="DATE_OBS",
)

offset = 0
while True:
    page = hqi.select(
        "VIEW_SP_HQI",
        what="DATE_OBS, FEAT_AREA_DEG2",
        where={"DATE_OBS": ("2012-06-01", "2012-06-02")},
        order_by="DATE_OBS",
        limit=1000,
        offset=offset,
    )
    ...  # process page.rows
    if len(page) < 1000:
        break
    offset += 1000

response = hqi.select("VIEW_AR_HQI", what="DATE_OBS, R_SUN", limit=3)
response.save("result.xml")
data = response.to_dict()  # {"description", "info", "tables": [{"name", "fields", "rows"}]}
table = response.to_astropy()

from pyhfc.api import HQIError

try:
    hqi.select("VIEW_AR_HQI", where="NO_SUCH_FIELD = 1")
except HQIError as err:
    print(err)

DEVELOPMENT

Install the git hooks (ruff, mypy, gitleaks…):

uv run pre-commit install

Run the checks and the tests:

uv run pre-commit run --all-files
uv run pytest

Tests querying the live HFC web service are deselected by default, run them with:

uv run pytest -m network

The continuous integration runs the checks and the tests on Python 3.12, 3.13 and 3.14, then builds the package. It is available for GitLab (.gitlab-ci.yml) and GitHub Actions (.github/workflows/ci.yml), and is triggered from the web interface, through the API, or when a tag is pushed. On GitHub, it can be triggered through the API with a workflow_dispatch event, or a repository_dispatch event of type ci:

gh workflow run ci.yml --ref develop

The development is done on the develop branch; the master branch only receives the tagged releases of the package.

The badges at the top of this file give the status of the last GitHub Actions run on the develop branch, the coverage of the tests on this branch, the tested Python versions and the licence. The coverage badge is updated by the CI runs on develop, which push a coverage.json file to the badges branch. The Python versions of the badge must be the ones of the pyproject.toml classifiers and of the CI matrices (this is checked by the tests). On GitLab, the pipeline and coverage badges can be added in Settings > General > Badges.

RELEASES

The development is done on develop; the master branch only receives the releases. To publish a new version of pyhfc on PyPI:

1. update the version (e.g. uv version --bump minor) and the CHANGELOG.rst on develop;

2. merge develop into master;

3. tag the merge commit on master with the version, and push the tag:

   git tag v0.3.0
   git push origin v0.3.0

The CI then runs the checks and the tests, builds the package and publishes it (publish-pypi job). The publication is refused if the tag (X.Y.Z or vX.Y.Z) does not match the version of the package, or if the tagged commit is not on master (see ci/check-release.sh). The files already on PyPI are skipped, so the GitHub and GitLab pipelines can both run on the same tag.

The publication uses the trusted publishing of PyPI: no token is stored in the CI, PyPI trusts the CI pipelines declared for the project. This must be configured once on PyPI:

• before the first release (the pyhfc project does not exist yet on PyPI), add a pending publisher in Your account > Publishing, with the project name pyhfc and, for GitHub: owner HELIO-HFC (the organization, not a user), repository PyHFC (the name only, not its url), workflow ci.yml, environment pypi; the first release then creates the project;

• for GitLab, add a publisher in the Publishing settings of the pyhfc project once it exists (or as the pending publisher, if the first release is made from GitLab), with the GitLab namespace and project, the top-level pipeline file .gitlab-ci.yml and the environment pypi.

On GitHub, protection rules (e.g. a required reviewer) can be added to the pypi environment in Settings > Environments, to approve each publication.

LICENSE

PyHFC is distributed under the European Union Public Licence v1.2 (EUPL-1.2), see the LICENSE file.

CONTACT

xavier.bonnin@obspm.fr