canvod-airflow
Apache Airflow DAGs that run the canVODpy pipeline every day. canvod-airflow is one of the
canVODpy extensions, optional packages
for canVODpy, an open Python ecosystem for
GNSS-Transmissometry (GNSS-T) canopy VOD retrievals.
What it does
canvod-airflow wraps the task functions of canvodpy run
(canvodpy.workflows.tasks) into Airflow TaskFlow DAGs. No pipeline logic
lives here: every task calls canvodpy.workflows.tasks, which has no
Airflow dependency of its own.
| DAG | Trigger | Tasks |
|---|---|---|
canvod_{site} |
@daily |
t_validate_dirs → t_wait_for_data (→ t_wait_for_sp3) → t_process_day → t_validate_ingest → t_calculate_vod |
canvod_backfill |
manual (params site, start_date, end_date) |
t_resolve_dates → t_process_day, one mapped task per day |
There is one daily DAG per site of the canvodpy settings, created when
Airflow parses the DAG file. t_wait_for_sp3 runs only when the settings use
agency orbit and clock products instead of broadcast ephemerides.
Install
uv add canvod-airflow
This installs apache-airflow>=3.3.2 too, the first release without
known vulnerabilities; an installed Airflow older than that is upgraded. canVODpy requires Python 3.14.
Airflow runs on POSIX systems (Linux, macOS); on Windows, run it in WSL2
(Airflow's prerequisites).
Deployment
Point Airflow's dags_folder at a module that imports both DAG modules
(Airflow only parses files that contain the words "airflow" and "dag"):
# <dags_folder>/canvod_dags.py
from canvod.airflow.daily_processing import * # noqa: F403 (airflow dag)
from canvod.airflow.backfill import * # noqa: F403
Create the canvod_store_write pool with one slot. Every task that writes
to a store (t_process_day, t_calculate_vod) runs in it, so commits to an
Icechunk store happen one at a time across all DAGs and backfill runs:
airflow pools set canvod_store_write 1 "Serialize Icechunk commits"
Documentation
airflow documentation and the canVODpy documentation.
Citation
Please cite the canVODpy extensions with their Zenodo DOI, 10.5281/zenodo.21359005, and canVODpy with 10.5281/zenodo.18496233.
License
1 maintainer
Metadata
1 maintainer
Release files for canvod-airflow 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| canvod_airflow-1.0.0.tar.gz | 11.5 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| canvod_airflow-1.0.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 24.8 kB
Release files / canvod_airflow-1.0.0.tar.gz
| Download URL | canvod_airflow-1.0.0.tar.gz |
|---|---|
| Size | 11.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
700087a3feb3dad743c8335e71b7d820a899216180484bc8da2c3a9288f813f9
|
|
BLAKE2b-256 checksum How to use checksums |
c75ce01302307c622d6db8430983fbd5de8b4340f7c2b8c38495bf954a4cb270
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / canvod_airflow-1.0.0-py3-none-any.whl
| Download URL | canvod_airflow-1.0.0-py3-none-any.whl |
|---|---|
| Size | 13.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b1cb0cebd7fa9efd1fe944ba6cb0901fa8c9dbdcf61ec331a16e8162b67d76a9
|
|
BLAKE2b-256 checksum How to use checksums |
e57fc939013ca1e61aa0e9fff09b887b3f442c3010d20c33a39739e70cd3d18a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for canvod-airflow 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
